The Definition of Regression to the Mean

Regression to the mean is the tendency for an unusually high or low measurement to be followed by one closer to the average, simply because the two measurements are not perfectly linked.

You meet it whenever something is picked for being extreme and then measured again: the worst week on a sales dashboard, the bottom tenth of a ranking, the patients with the highest readings, the variant that looked best on day one of an A/B test. The second measurement usually looks less extreme. What the concept decides is whether that change gets credited to whatever you did in between, or to nothing at all.

The rule fits in one line: whenever two measurements correlate at less than 1, a group chosen for being extreme on the first will, on average, be less extreme on the second. No cause is needed for that to happen.

The term comes from Francis Galton's 1886 paper "Regression towards Mediocrity in Hereditary Stature," published in the Journal of the Anthropological Institute of Great Britain and Ireland. "Mediocrity" meant average, not poor.

Galton compared the heights of adult children with the average height of their two parents, which he called the mid-parent height. According to the BMJ's account, he measured 930 adult children; other accounts describe the dataset as 1,078 families. Either way, the pattern was the same. Children of taller-than-average parents tended to be shorter than their parents, and children of shorter-than-average parents tended to be taller.

The size of the pull was about two-thirds. As Wikipedia summarizes it, a child's height typically sat about two-thirds as far from the population average as the mid-parent height did. Say a pair of parents averages 3 inches above the mean: their children would be expected to land about 2 inches above it. Still tall, just less so.

Galton first read this as a biological force, a "reversion" toward ancestral type. What changed his mind, according to a history in Studies in History and Philosophy of Science, was that the effect ran both ways. Pick unusually tall children and their parents turn out to be closer to average than they are. A one-way inherited force cannot explain that. Imperfect correlation can.

That is the mechanism. Any measurement is part stable signal (true height potential, real skill, genuine demand) and part things that do not repeat (luck, measurement noise, a good day). An extreme result usually has both pushing the same way. On the next measurement the stable part stays and the luck is redrawn, so the result drifts back.

The correlation coefficient, written r or ρ, measures how tightly two sets of numbers move together, from 0 (not at all) to 1 (perfectly). Researchers writing in BMC Medical Research Methodology state the condition directly: if a first measurement is above the mean, the second is expected to be smaller "if only 0 ≤ ρ < 1." Only a perfect correlation of exactly 1 removes the effect, and real measurements of people almost never have one. When both measurements share the same spread, the expected distance from the mean on the second measurement is the first distance multiplied by the correlation. Galton's two-thirds was that multiplier for height.

Flight instructors and praise. Daniel Kahneman, recounting the episode in his 2003 Nobel autobiography, was teaching Israeli Air Force instructors that praise works better than punishment. A veteran instructor objected: cadets he criticized after a bad landing usually did better next time, and cadets he praised after a great landing usually did worse. Kahneman saw that the feedback was doing nothing. Exceptional landings were followed by ordinary ones regardless. He later showed the same pattern with people throwing coins at a target with no feedback at all: the best first throwers did worse on the second. His summary, quoted in the Journal of the Experimental Analysis of Behavior:

A walking program's lowest group. A study of a health program for older lesbian women reported a "marked step increase" among participants who started in the lowest third of daily step counts, and recommended the program for senior and LGBT centers. In a letter to the Journal of Women & Aging, David Allison argued that the improvement was substantially, possibly entirely, regression to the mean. Selecting the lowest baseline group almost guarantees a rebound at follow-up.

Hospital league tables. The BMJ article notes that giving extra resources to the worst-ranked hospitals tends to make them look improved, while top performers given the same help tend to slip in the rankings. Anyone who wants to show results from a policy gets more apparent improvement by targeting the bottom of the table.

The shared word causes real confusion, because Galton's work also gave rise to regression analysis, the line-fitting technique behind many forecasts. The two are related in their math but answer different questions. A second confusion is treating the drift back as a force, the way Galton first did.

The misreading to watch for is treating a return toward average as proof that something worked, or that something failed. Take a hypothetical call center that sends its bottom-ranked agents to a coaching program and then celebrates their improved scores.

So when a group was picked because it was extreme, ask one question before giving credit or blame: compared with what? A similar group selected the same way and left alone, a control group, shows how much movement regression alone produces. Without one, the before-and-after number cannot separate the program from the drift. For the same idea worked through with a fuller set of figures, see regression to the mean with real numbers.

The same check applies in reverse. If last quarter's top performer slips after a bonus, the bonus probably did not make them complacent, any more than praise ruined Kahneman's cadets.