Correlation vs Causation: The Three Explanations for Any Pattern

1. The three explanations, and the quiet fourth

Before the explanation: a dashboard shows that customers who open your emails spend more. Marketing wants to double the email volume. What are the other ways that pattern could have happened, without email causing anything?

A correlation means two things move together. When X is high, Y tends to be high, or tends to be low. That is the whole fact. The question that matters is why, and there are exactly four candidate stories.

• X causes Y. Opening emails makes people spend more. This is the story everyone wants.
• Y causes X. People who already spend a lot are more interested, so they open more emails. Same data, arrow reversed, completely different action.
• A third thing drives both. Loyal customers both open emails and spend. Loyalty is the engine, and email opening is just along for the ride. The third thing has a name: a confounder, which in everyday words is the hidden common cause that makes two innocent things look connected.
• Coincidence. In a small sample, unrelated things line up by luck alone. This one is quiet because nobody proposes it in a meeting, and it is the most common story of all when the sample is small. Section 7 puts a number on it.

The habit this page is selling costs about ten seconds. Before acting on any pattern, say all four stories out loud, in order, and notice which ones you cannot rule out. Most patterns die at story three, and the ones that survive all four are the ones worth an experiment.

2. The worked example: a real correlation you can compute

Before the explanation: ice cream sales and drownings rise and fall together, month by month, in almost every summary of this classic. Nobody believes ice cream drowns people. So what is the third thing?

Here is a full year for one lake town. Twelve rows, three columns: average temperature, ice cream sales in thousands of dollars, and drownings.

Read the sales column and the drownings column together. July: highest sales, most drownings. January and December: lowest sales, almost none. The two columns clearly move together, and the correlation between them computes to 0.86. Here is the code that produces every correlation on this page, so you can rerun it yourself.

import pandas as pd

df = pd.DataFrame({
    "month": ["Jan","Feb","Mar","Apr","May","Jun",
              "Jul","Aug","Sep","Oct","Nov","Dec"],
    "avg_temp_f":  [36,38,46,55,65,74,81,79,70,58,47,39],
    "ice_cream_k": [8,9,14,19,27,35,42,40,30,21,14,10],
    "drownings":   [1,0,0,1,4,3,5,4,5,0,1,0],
})

print(df[["avg_temp_f","ice_cream_k","drownings"]].corr().round(2))

             avg_temp_f  ice_cream_k  drownings
avg_temp_f         1.00         0.99       0.86
ice_cream_k        0.99         1.00       0.86
drownings          0.86         0.86       1.00

Now read the whole grid, not just the one cell. Ice cream and drownings sit at 0.86. But temperature correlates with ice cream at 0.99 and with drownings at 0.86. The third thing did not need hunting. It is right there in the output, correlated with both suspects more strongly than they need to be correlated with each other. Heat sends people to the ice cream stand and heat sends people into the lake, and the two crowds never have to meet.

Picture running that three-column grid on your own data: your metric, the thing it moves with, and the one variable you quietly suspect is behind both. The confounder usually announces itself in exactly this way, as the row with the biggest numbers in it.

3. What a correlation coefficient actually measures

Before the explanation: the number 0.86 came out of the machine. What is it a measurement of, and what would 1.0, 0, and negative 0.86 each look like?

The correlation coefficient, written r, measures linear co-movement: how tightly two columns track a straight line together. It runs from −1 to 1. At 1, knowing one value tells you the other exactly, and both rise together. At −1, the same, but one falls as the other rises. At 0, knowing one tells you nothing linear about the other. Our 0.86 means the monthly pairs sit close to a rising line, with some scatter.

Three limits on what r says, each of which matters in real work.

• It is direction-blind. r(sales, drownings) and r(drownings, sales) are the same number. The statistic has no idea which column you think is the cause. Any arrow you see in it, you drew yourself.
• It only sees straight lines. A perfect U-shaped relationship, where Y is high at both extremes of X, can produce an r near zero. A near-zero r means "no linear pattern", not "no pattern". This is one reason to plot the points before trusting the number.
• It says nothing about size. An r of 0.9 can describe an effect too small to matter, because r measures tightness of tracking, not steepness. A tight pattern of tiny changes still scores high.

The word for a variable's r with a business outcome is often spoken as if it were a strength-of-cause score. It is a tightness-of-tracking score, computed with no knowledge of cause at all. Keeping those two ideas separate is most of this page.

4. The cheap test: hold the third thing still

Before the explanation: if temperature really is the engine behind both columns, what should happen to the ice cream and drownings correlation among months that all have roughly the same temperature?

It should shrink toward zero. If heat is doing all the driving, then among equally hot months there is no driver left, and sales and drownings should wander independently. That prediction is testable on the data you already have, and this is the cheapest causal check an analyst owns: segment by the suspected confounder and see whether the pattern survives.

warm = df[df.avg_temp_f >= 65]   # May through Sep, 5 months
cool = df[df.avg_temp_f < 65]    # the other 7 months

print(round(warm.ice_cream_k.corr(warm.drownings), 2))
print(round(cool.ice_cream_k.corr(cool.drownings), 2))

0.1
0.02

The correlation was 0.86 across the whole year. Inside the warm months it is 0.10. Inside the cool months it is 0.02. Once temperature is held roughly still, the relationship between ice cream and drownings is gone. The pattern did not survive the split, which is exactly what "a third thing drives both" predicts and what "X causes Y" does not.

Say in your own words why the correlation vanishing inside each band clears ice cream, before reading on. If you can say it, you own the method.

Here is the version I would say. The yearly correlation was built out of one contrast: warm months have high sales and high drownings, cool months have low sales and low drownings. Warm months average 4.2 drownings against 0.43 in cool months. Remove that contrast by comparing only like with like, and there is nothing left, so the contrast, meaning temperature, was the whole relationship.

5. Reverse causation, and why timing settles it

Before the explanation: story two was "Y causes X". What is the one thing a cause must always do that its effect cannot?

Come first. An effect cannot happen before its cause. So the standard check on reverse causation is timing: line the two series up in time and ask which one moves first.

Take the email example from section 1. If opening emails causes spending, the open should come before the purchase, and this month's opens should predict next month's spending better than the reverse. If the data shows spending rising before open rates rise, the arrow you wanted is dead on arrival. In practice this means computing the correlation at a lag: shift one column by a month and recompute. It is the same corr() call on a shifted column, so it costs nothing.

Two honest cautions. First, timing can eliminate an arrow but cannot confirm one, because a confounder can also move both series in sequence. Cold snaps end, then ice cream recovers, then swimming recovers. Second, some pairs feed each other in both directions, like practice and confidence, and a lagged correlation will show both arrows at once. When that happens, write down both, because that is the true shape.

6. The full before and after

Same data, same finding, written up twice. The difference is what a reader is licensed to do after reading it.

Before

Ice cream sales are strongly correlated with drownings
(r = 0.86). Reducing summer ice cream promotions should
be considered as a drowning-prevention measure.

Every word before the parenthesis is defensible, and the sentence after it is fiction. The number is real. The leap from "correlated" to "reducing X will change Y" is the entire mistake, and nothing in the data supports it.

After

Monthly ice cream sales are associated with drownings
(r = 0.86, n = 12 months).

Checked: temperature is a confounder. It correlates with
both series (0.99 with sales, 0.86 with drownings), and
within temperature bands the association disappears
(r = 0.10 warm months, r = 0.02 cool months).

Read: both rise in hot months because heat drives both.
No causal claim. If prevention budget depends on this,
the variable to act on is warm-month lake supervision,
not sales.

Three habits are visible in the after version. It says "associated with", which claims co-movement and nothing more. It names the confounder it checked and shows the check, which is what makes the write-up trustworthy rather than merely cautious. And it states the sample size, so the reader can weigh the coincidence story for themselves. Naming what you checked and what you could not check is the same discipline as documenting data limitations, applied to one sentence.

7. Edge cases that make correlations lie

Before the explanation: with only 12 rows, how often would two completely unrelated columns show a correlation as strong as 0.5, just by luck?

More often than feels possible. I generated 1,000 pairs of random 12-value columns, pure noise with no relationship at all, and counted: 106 of the 1,000 pairs hit an r of 0.5 or stronger, and 10 of them hit 0.7 or stronger. Roughly one random pair in ten looks moderately correlated at this sample size. That is the coincidence story with a number on it, and it is why an exciting correlation in a 12-row summary deserves suspicion before celebration.

import random
random.seed(11)
hits = 0
for _ in range(1000):
    x = pd.Series(random.random() for _ in range(12))
    y = pd.Series(random.random() for _ in range(12))
    if abs(x.corr(y)) >= 0.5:
        hits += 1
print(hits)   # 106

Four more ways a real correlation misleads, each named.

Both series just trend. Anything that grows over time correlates with anything else that grows over time: revenue, headcount, and the number of guides on this site would all inter-correlate beautifully. A correlation between two trending series is mostly a correlation with the calendar. Compare month-to-month changes instead of levels before believing it.

The pattern flips when you split it. A relationship can point one way in every subgroup and the opposite way overall, because group sizes differ. This is Simpson's paradox, and it is the same segmenting move from section 4 delivering a nastier surprise: the aggregate answer and every subgroup answer can genuinely disagree. It is why segmenting by the variables you know matter is not optional.

You only kept the survivors. Correlations computed on a filtered group, like top customers or games with many reviews, can be created or destroyed by the filter itself. If both variables helped a row pass the filter, the survivors will show a link the full population does not have.

One outlier is holding the whole number up. With few rows, a single extreme pair can manufacture a strong r on its own. Plot the points once before quoting the number. If the story collapses when one dot is covered with your thumb, it was never a story. Plotting before computing is the core move of exploratory data analysis.