Correlation vs. Causation — The Difference Matters. A Lot. | In The Money by Zerodha

ELI5/TLDR

Two things can move up and down together perfectly without one having anything to do with the other. A man once built a model that “predicted” the US stock market with 99% accuracy using butter and sheep — pure coincidence dressed up in clean math. This episode walks through why moving-together (correlation) is not the same as one-thing-driving-the-other (causation), and gives you two practical filters for any market pattern: is there enough time gap to actually trade it, and are you sure you have the cause and effect pointing the right way.

The Full Story

The butter that predicted the stock market

The episode opens with a true story. In 1995 a quant named David Leinweber went hunting through a UN data CD-ROM covering 145 countries, deliberately looking for patterns with no logic behind them. He found that butter production in Bangladesh tracked 75% of the S&P 500’s movement over a decade.

Not GDP numbers, not earnings, not interest rates, but butter in Bangladesh.

He kept going. Add US butter and cheese, and the fit jumped to 95%. Throw in the sheep population of Bangladesh and the US, and he hit 99% — a near-perfect prediction of the world’s most-watched stock index, built entirely from dairy and livestock. The math was spotless. The meaning was zero. He eventually published it as a joke paper titled “Stupid Data Miner Tricks.”

The Indian version of the same lesson is what the host calls the panwala indicator: when your local pan-shop owner starts handing out stock tips, the market has probably topped. Funny, with a grain of truth, but not something you’d bet money on.

Where the word “correlation” came from

The term has a strange origin. Francis Galton — a Victorian scientist and Charles Darwin’s cousin — was trying to prove that genius runs in families. He measured fathers and sons, plotted height against forearm length, head width, and so on, and kept noticing the same thing: when one measurement went up, the other tended to go up too, but neither was causing the other. Both were just downstream of the same hidden cause, genetic inheritance.

He needed a word for “two things move together but neither drives the other,” and that’s how correlation was born. His student Carl Pearson turned it into a single number, the correlation coefficient — running from -1 to +1.

Think of it as a togetherness score. +1 means two things move in perfect lockstep in the same direction. -1 means perfect lockstep in opposite directions. 0 means no relationship at all (the example given: shoe size and test scores). Nifty50 and Sensex score near +1, because they’re two slightly different baskets of the same large Indian companies reacting to the same news.

The crucial bit: the coefficient is blind to cause. As Pearson noted with some delight, the number doesn’t know and doesn’t care why two things move together.

The hidden third thing

The cleanest example in the episode: every Indian summer, ice cream sales rise and power cuts rise. Plot them and you’d see a near-perfect correlation. Does ice cream cause blackouts? Obviously not. The heat drives both — people eat more ice cream and run their ACs harder, straining the grid.

That hidden driver doing the real work — summer heat here — is called a confounding variable. The lurking cause that the two visible variables can’t see. Most spurious correlations are exactly this: two symptoms of one unseen cause.

Causation is the stronger claim. It says a change in X genuinely drives a change in Y, with an actual mechanism connecting them. Rain causes wet ground. An RBI rate hike causes borrowing costs to rise. There’s a chain of logic, not just a shared graph.

Three levels of understanding cause

The host borrows Judea Pearl’s framework from The Book of Why, called the ladder of causation — three rungs, each more powerful than the last.

Rung one, association. When I see X, how likely am I to see Y? This is just the correlation coefficient. Pearl points out, a little uncomfortably, that this is also all most AI systems do today — sophisticated pattern matching, nothing more.

Rung two, intervention. You actively change something and watch what happens. This is the randomized controlled trial — the gold standard in medicine. Split a large group in two at random, give one a treatment and the other a placebo, measure the difference. The random split cancels out every confounding variable at once.

Rung three, counterfactual reasoning. Asking what would have happened if things had been different. My headache went away after a pill — but would it have gone away anyway? Going back to the market superstition: if there were no Amavasya this weekend, and the market still fell on Friday, does Monday’s gap-down become any less likely?

The catch for markets: rung two is impossible. You can’t randomly assign one set of investors a rate hike and another a rate cut. Markets have exactly one history. So traders are almost always stuck on rung one — staring at patterns, asking what they mean, with no clean way to test cause. And humans hate not knowing why, so we grab the nearest story, or invent one.

Two filters that actually matter at the desk

Even when a relationship is real, two things decide whether it’s useful.

The first is lag — the time gap. Suppose asset B reliably moves right after asset A. If that gap is one microsecond, the correlation is real but worthless to you; by the time you see it and reach for the buy button, it’s gone. Only high-frequency firms with servers wired into the exchange can play that. For a pattern to be tradable, the gap has to be wide enough for a human (or at least an automated system) to act.

The relationship is real but the opportunity is not tradable.

This is why pair trading works on some timeframes and falls apart on others — if two drifting assets snap back together in seconds, there’s no window.

The second is causal direction — which way the arrow points. Here the host makes the sharpest argument of the episode, aimed at technical indicators. RSI, MACD, moving averages — every one of them is calculated from price. Price is the input, the indicator is the output.

So the arrow can only run one way: price moves the indicator, never the reverse. When RSI hits 30 and traders say “oversold, expect a bounce,” they’ve quietly flipped the arrow. RSI is at 30 because price already fell sharply. The indicator isn’t predicting anything; it’s reporting what already happened.

They are the market’s shadow, not its sun.

Pearl’s name for getting the arrow backwards is reverse causality. The host’s image: a dog walking under a bullock cart, convinced the cart moves because of him. It doesn’t mean indicators are useless — only that they’re a rear-view mirror, a picture of what price has already done, not an instruction for what it will do next.

Back to the superstitions

The episode bookends with two social-media-style claims it flashed at the start — about Amavasya falling on a weekend forcing a Monday gap-down, and five days of negative FII flows guaranteeing a bounce. The point lands without needing to debunk each one: you now know how not to read them. If you can test such a claim, test it. And if you find a gorgeous correlation with no mechanism, no usable lag, or a backwards arrow — you’re better off ignoring it.

Key Takeaways

Spurious correlation is guaranteed, not rare. Search enough variables and you will find something that fits beautifully and means nothing — butter and sheep “predicting” the S&P at 99% is the proof.
Correlation coefficient = a single togetherness score from -1 to +1. +1 is perfect same-direction lockstep, -1 perfect opposite, 0 no relationship. It is completely blind to cause.
Confounding variable = a hidden third thing driving two visible ones (summer heat behind both ice cream sales and power cuts). Most fake correlations are this.
Causation needs a mechanism — an actual chain of logic (rate hike → higher borrowing costs), not just a shared graph.
The ladder of causation (Judea Pearl): rung 1 association (correlation, what AI does), rung 2 intervention (the randomized trial that cancels confounders), rung 3 counterfactual (“what if it had been different”).
Markets are stuck on rung one — one history, no possible controlled experiment, so cause is rarely provable. We fill the gap with invented narratives.
The lag test: even a real relationship is useless if the time gap is too small to act in. Microsecond edges belong to co-located HFT servers, not to you.
The direction test (reverse causality): make sure the arrow points the right way. RSI, MACD, moving averages are all derived from price — price drives them, never the reverse. Indicators are a rear-view mirror.
Two questions for any pattern: (1) is the lag wide enough for me to act? (2) am I sure I have cause and effect the right way round? Get either wrong and even a genuine relationship costs you money.

Claude’s Take

This is a clean, well-told primer that does one thing and does it well: it takes a stats-101 distinction everyone nods along to and shows why getting it wrong actually drains an account. The butter-and-sheep opener is a real and famous example (Leinweber’s “Stupid Data Miner Tricks” is genuine), and the host resists the temptation to overclaim — he explicitly jokes about the fake ones so you don’t mistake them for data.

The strongest section is the indicators-as-shadow argument. It’s correct and underappreciated: RSI, MACD and moving averages are arithmetic functions of past price, so treating them as a cause is a genuine logical error, not just a stylistic preference. The lag point is equally practical and rarely made this plainly — plenty of “edges” are real and simply too fast to touch.

Where it’s a touch loose: “even if you see a perfect correlation, you’re better off ignoring it altogether” overshoots. Correlation is the starting point of nearly all useful analysis — the warning should be against acting on it without a mechanism, not against looking at it. And the Daniel Pearl aside about Judea Pearl is true but pure trivia. Minor quibbles in an otherwise honest, jargon-light explainer. Seven out of ten: nothing novel for anyone who’s read Pearl or Wheelan, but a genuinely good 20 minutes for someone who hasn’t, and the book recommendations at the end are exactly the right three.