The central limit theorem, from first principles

The normal distribution is suspiciously popular. It shows up in measurement error, in test scores, in the sampling distribution of almost every average anyone computes. That is odd, because nothing in nature has any obligation to be bell-shaped. Coin flips are not bell-shaped. Waiting times are not bell-shaped. Order values are usually a long right tail with a fat clump near zero.

The central limit theorem is the explanation. It says the bell curve is not a fact about the world. It is a fact about adding. Add up many small independent effects, and the sum forgets what the individual effects looked like. Whatever went in, a bell curve comes out.

That “forgetting” is the theorem’s real content. Build it up carefully, because the theorem is quoted far more often than it is understood — usually in the form of a rule about the number 30 that the theorem never states.

What the theorem actually says

Take a quantity you can measure repeatedly. Each measurement is a draw from some distribution — any distribution, as long as it has a finite mean and a finite variance. Draw n of them independently, average them, and call that average x̄.

Two things happen as n grows:

• x̄ closes in on the true mean. That is the law of large numbers, and it is the boring half.
• The distribution of x̄ — how much it wobbles from sample to sample — becomes normal, centred on the true mean, with standard deviation σ/√n.

The second point is the theorem. Note precisely what it is about. It is not about your data. It is about a statistic computed from your data. Your raw measurements stay as skewed and lumpy as they always were; collecting more of them will never make a histogram of income look like a bell. What becomes normal is the sampling distribution of the average.

This is the single most common misreading, and it matters. “I have 50,000 rows, so my data is normal enough” is wrong. “I have 50,000 rows, so the mean of those rows has a nearly normal sampling distribution” is right, and is what lets you put an interval around it. Four panels make the difference visible.

The population never changes shape — it is skewed at every sample size. What changes is the distribution of the average. At n = 1 the average is a single draw, so it is exactly the population. By n = 30 it is a narrow bell around the true mean.

Why adding smooths everything out

The formal proofs go through characteristic functions and are not much help to intuition. The mechanical picture is better.

Adding two independent random quantities convolves their distributions. Slide one distribution across the other, multiply, and sum. Convolution is a smearing operation: every spike gets spread out by the shape of the thing it is added to. Do it once and sharp corners round off. Do it repeatedly and the shape drifts toward the one distribution that convolution leaves unchanged — the Gaussian.

You can watch this with dice. A single die is flat: six equal bars. Two dice already form a triangle. Three dice look bell-ish. By five, a normal curve drawn over the histogram is hard to distinguish from it. Nobody made the dice normal. The addition did.

import numpy as np

rng = np.random.default_rng(0)

# a hard case: a coin flip is as far from a bell as it gets
flips = rng.integers(0, 2, size=(200_000, 30))
means = flips.mean(axis=1)

print(means.mean(), means.std())          # ~0.5, ~0.091
print(0.5 / np.sqrt(30))                  # 0.0913 — the predicted σ/√n

The predicted spread matches the observed spread. That σ/√n is the practical payoff of the theorem: quadrupling the sample halves the uncertainty in the mean, no matter what shape the underlying data has.

It also explains why so many natural quantities look normal. Height is the sum of many small genetic and environmental contributions. Measurement error is the sum of many small independent nudges. Where effects multiply rather than add — incomes, city sizes, file sizes — you get a log-normal instead, because the logs add. Same theorem, different arithmetic.

Where it stops working

The theorem has conditions, and each one fails in real data.

The last row is the one that bites in production. Convergence speed depends on skewness, not on n alone. A symmetric distribution converges after a handful of draws. A distribution where most of the total sits in a rare tail — revenue, insurance claims, click-to-purchase value — may need tens of thousands.

Where “n > 30” came from

It came from textbooks that needed a cutoff for a printed t-table, and it survived because it is easy to remember. The theorem contains no 30. It contains a limit as n goes to infinity, and a rate of approach that depends on the shape of the data.

For coin flips, 30 is generous. For daily revenue per customer, 30 is a joke. The honest version of the rule is: the number you need depends on how skewed your data is, and you can check rather than guess.

Checking is cheap. Resample your own data, recompute the statistic a few thousand times, and look at the resulting distribution directly. If it is symmetric and bell-shaped, the normal approximation is fine and you have just confirmed it instead of assuming it. If it is skewed, use the resampled percentiles and skip the approximation entirely. That is exactly what the bootstrap does, and it is why the bootstrap is the better default for anything other than a plain mean.

The smoothing that produces the bell curve is animated in But what is the Central Limit Theorem? by 3Blue1Brown.

One boundary is worth holding onto, because it is where the theorem quietly stops covering you. It is about sums and averages. A maximum, a minimum, a ratio of two noisy quantities, a 99th percentile — none of those are averages, and none of them get a bell curve for free, however many rows arrive. For those statistics the resampled distribution is the only cheap answer available, and the normal approximation was never on offer.