← All Kits · SQL Kit · Statistics Kit
Why Exploratory Data Analysis Matters
Here is the most common way a data analysis goes wrong, and it has nothing to do with math: the analyst computes an answer from data they never actually looked at. The numbers come out precise, confident, and wrong — because the data had a quirk nobody checked for. Exploratory data analysis (EDA) is the discipline that prevents this. This guide explains what it is, the classic research showing why it's non-negotiable, and what it looks like in real projects you can open and read.
- Detective work vs courtroom work
- Where EDA came from (and why a famous statistician rebelled)
- The proof that summaries lie: a quartet and a dinosaur
- What exploration looks like in a real project
- The habits, distilled
- The honest limit: exploration finds suspects, not verdicts
- References
Detective work vs courtroom work
Data work has two distinct modes, and mixing them up causes most analytical failure:
| Exploratory (detective) | Confirmatory (courtroom) | |
|---|---|---|
| Question | "What is going on here?" | "Is this specific claim true?" |
| Attitude | Open, skeptical, curious | Formal, rule-bound, pre-committed |
| Output | Hypotheses, surprises, data-quality discoveries | Evidence for or against one hypothesis |
A detective doesn't walk into a crime scene with a verdict; they look at everything, notice what's odd, and form theories. Only later does the courtroom test one theory formally. Exploratory analysis is the crime-scene phase — and skipping it means prosecuting a case you never investigated. The comparison isn't a modern teaching gimmick: John Tukey, who founded the field, framed exploratory work as inherently detective-like, and the research literature has used the analogy ever since (Behrens, 1997).
Where EDA came from (and why a famous statistician rebelled)
In 1962, John Tukey — a Princeton mathematician who also coined the words "bit" and "software" — published a 67-page argument called The Future of Data Analysis that scandalized his own field. Statistics, he argued, had become obsessed with the courtroom: elegant formal tests, applied to questions someone had already decided to ask. Almost nobody was studying the harder, earlier problem: how you figure out what questions the data wants you to ask. He proposed treating data analysis as its own empirical science, one that starts by looking (Tukey, 1962).
Fifteen years later his book Exploratory Data Analysis (1977) turned the argument into a toolkit — plots, summaries, and residual-checking habits designed to make the unexpected visible. Its most quoted line is the mission statement of the whole field:
"The greatest value of a picture is when it forces us to notice what we never expected to see." (Tukey, 1977)
Note what that sentence actually says: the payoff of looking is not confirming what you expected — it's being ambushed by what you didn't. Every technique in EDA is a machine for manufacturing productive ambushes.
The proof that summaries lie: a quartet and a dinosaur
Why isn't a table of averages enough? In 1973, statistician Francis Anscombe settled this with four small datasets, now called Anscombe's quartet. All four share nearly identical summary statistics — same average of x, same average of y, same variance, same correlation (0.816), same fitted trend line (Anscombe, 1973). By the numbers they're quadruplets. Graphed, they're strangers:
- One is a healthy, ordinary linear relationship — what the summary implies.
- One is a smooth curve — the linear trend line is simply the wrong model.
- One is a perfect line except for a single outlier that dragged the trend off course.
- One is no relationship at all — a vertical stack of points plus one extreme value that manufactured the entire correlation.
Four different realities, one identical scorecard. If your analysis stops at the scorecard, you cannot tell which reality you're in.
In 2017, researchers Justin Matejka and George Fitzmaurice pushed the point to its comic extreme: they showed you can gradually nudge a dataset's points into nearly any picture — including a hand-drawn Tyrannosaurus rex — while keeping mean, standard deviation, and correlation identical to two decimal places. Their "Datasaurus Dozen" is twelve wildly different plots (a dinosaur, a star, concentric circles...) with matching summary statistics (Matejka & Fitzmaurice, 2017). The lesson is Anscombe's, updated for the modern era: never trust summary statistics alone; always visualize.
What exploration looks like in a real project
The classics prove why; here is what it looks like, from open portfolio projects you can read end to end. None of this is glamorous. All of it changed the results.
Peek before you trust: the phantom-column catch
Rule one of EDA is embarrassingly simple: look at actual rows before computing anything. In a project analyzing Billboard chart history, the very first look at the imported table showed two mystery columns the source file didn't have — and song titles chopped in half. Cause: a wrong import setting had split every title containing a comma (like "Let It Snow, Let It Snow, Let It Snow") across columns. Every count run on that import would have been silently wrong. Cost of catching it: one SELECT * ... LIMIT 20 and thirty seconds of reading. The Streaming Hidden Gems project hit the same class of problem from another direction — its "CSV" files turned out to be tab-separated with no quoting at all, documented in the README's data-quality section.
Check the data against arithmetic it must obey
Good exploration tests expectations the data has no excuse to fail. A chart that lists 100 songs a week must have a row count divisible by 100; weeks-times-100 must equal total rows exactly. When a check like this passes, the import earns trust; when it fails, you just found a problem at the cheapest possible moment. The Steam Hidden Gems project and Streaming Hidden Gems project both carry a "Validation" section in their READMEs showing exactly these cross-checks — totals reconciled two independent ways before any conclusion was allowed to rest on them.
Interrogate the weird: why is a number stored as text?
Exploration means treating oddities as leads, not annoyances. In the Billboard data, a chart-position column imported as text instead of numbers. A detective doesn't shrug at that; the follow-up queries found the culprit — thousands of rows carrying the letters 'NA', plus the discovery that "debut week" was marked two different ways in different eras of the archive. Every later query had to speak both dialects. Miss that during exploration and every debut-related count afterward is quietly wrong. (The same investigation pattern — hypothesis, targeted query, verdict, next hypothesis — is walked through step by step in the SQL CASE expression guide, where exploring a messy artist-credit column ends up dictating the cleaning rule.)
Let distributions choose your thresholds
Analyses constantly need cutoffs: how many reviews make a rating trustworthy? How many votes make a movie "known"? The lazy move is picking a round number from the gut. The exploratory move is asking the data first: plot or tabulate the distribution, find where behavior actually changes, and pick a cutoff you can defend. Both gems projects do this on camera — Steam Hidden Gems derives its review-count floor from the measured distribution of review volumes, and Streaming Hidden Gems derives its vote-count band the same way, with the rejected alternatives documented. A threshold with a distribution behind it survives the question "why that number?" A guess doesn't.
The habits, distilled
| Habit | The move | What it catches |
|---|---|---|
| Look at rows | SELECT * ... LIMIT 20 before anything else | Import wrecks, swapped columns, mangled text |
| Count and reconcile | Row counts checked against arithmetic the data must obey | Silently dropped or duplicated rows |
| Profile every column | Distinct values, min/max, missing-value markers, type surprises | 'NA' strings, two markers for one meaning, text-typed numbers |
| Visualize, don't summarize | Plot distributions and relationships before trusting averages | Everything Anscombe and the Datasaurus warned about |
| Chase oddities | Treat every "huh, weird" as a lead with a follow-up query | The era-stitched archives and duplicate entities real data is full of |
| Derive thresholds | Cutoffs from measured distributions, never from vibes | Indefensible analysis choices |
The honest limit: exploration finds suspects, not verdicts
One warning keeps EDA honest. When you explore freely, you will find patterns — including patterns that are pure coincidence, because looking at data from enough angles guarantees something eventually looks interesting. The detective/courtroom split exists precisely for this: a pattern discovered during exploration is a suspect, not a conviction. It earns a hypothesis, which then needs testing on evidence that didn't generate it — fresh data, a held-out sample, or a formal test specified in advance. Treating an exploratory find as a proven fact is the analytical equivalent of convicting the first person the detective happened to interview. Explore boldly; conclude carefully. (And what exploration finds, the analysis must account for — that's its own discipline, covered in How Analysts Document Limitations.)
The SQL Kit teaches the query skills every exploration above uses, the Statistics Kit covers the distributions and correlations this guide leans on, and the Portfolio Projects page shows how to turn an explored dataset into a project worth showing an employer. All free, all in your browser.
Open the SQL Kit →References
- Tukey, J. W. (1962). The future of data analysis. The Annals of Mathematical Statistics, 33(1), 1–67. doi:10.1214/aoms/1177704711
- Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
- Anscombe, F. J. (1973). Graphs in statistical analysis. The American Statistician, 27(1), 17–21. doi:10.2307/2682899
- Behrens, J. T. (1997). Principles and procedures of exploratory data analysis. Psychological Methods, 2(2), 131–160. doi:10.1037/1082-989X.2.2.131
- Matejka, J., & Fitzmaurice, G. (2017). Same stats, different graphs: Generating datasets with varied appearance and identical statistics through simulated annealing. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 1290–1294. ACM.