The Central Limit Theorem

⚡ Key Takeaways

One-line statement: The sum (or average) of a large number of independent, identically distributed random variables — each with finite mean and variance — is approximately normally distributed, regardless of the underlying distribution. Key result in words: Properly centered and scaled, the sum of i.i.d. random variables converges in distribution to a standard normal $N(0,1)$ as the sample size grows without bound. Where it applies: Statistical inference, error analysis, polling, finance, signal processing, statistical mechanics — anywhere aggregate randomness emerges from many small independent contributions. Who/when: Foundations by Abraham de Moivre (1733) and Laplace (1810); the modern general form by Lyapunov (1901) and Lindeberg (1922); named "central" by George Pólya (1920).

💡 The Core Insight

Information about the shape of a distribution lives in its higher moments and in the fine structure of its characteristic function near the origin. When you average $n$ independent copies and rescale by $\sqrt{n}$ , the characteristic function gets raised to the $n$ -th power at a point shrinking like $1/\sqrt{n}$ . In this scaling limit only the first two moments survive: the mean determines the center, the variance determines the spread, and everything else vanishes. The Gaussian is the unique fixed point of this averaging-and-rescaling operation, which is why the bell curve is an attractor in the space of distributions.

DefinitionStandardization and Convergence in Distribution

Let $X_1, \dots, X_n$ be i.i.d. random variables with $\mathbb{E}[X_i] = \mu$ and $\operatorname{Var}(X_i) = \sigma^2 \in (0, \infty)$ . Define the sample mean $\bar{X}_n$ and the standardized sum $Z_n$ :

1

\quad \bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i, \qquad S_n = \sum_{i=1}^n X_i

Derivation hint: These are merely definitions; linearity of expectation gives $\mathbb{E}[\bar X_n]=\mu$ and $\mathbb{E}[S_n]=n\mu$ .

2

\quad Z_n = \frac{S_n - n\mu}{\sigma\sqrt{n}} = \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}

Derivation hint: Subtract the mean ( $\mathbb{E}[S_n]=n\mu$ ) and divide by the standard deviation ( $\operatorname{Var}(S_n)=n\sigma^2 \Rightarrow \mathrm{SD}=\sigma\sqrt n$ ) so that $\mathbb{E}[Z_n]=0$ , $\operatorname{Var}(Z_n)=1$ . A sequence $Z_n$ converges in distribution to $Z$ (written $Z_n \xrightarrow{d} Z$ ) if $F_{Z_n}(x) \to F_Z(x)$ at every continuity point $x$ of the limit CDF $F_Z$ .

TheoremClassical (Lindeberg–Lévy) Central Limit Theorem

Let $X_1, X_2, \dots$ be i.i.d. with mean $\mu$ and finite, positive variance $\sigma^2$ . Then

3

\quad Z_n = \frac{1}{\sigma\sqrt{n}}\sum_{i=1}^n (X_i - \mu) \xrightarrow{\ d\ } N(0,1)

Derivation hint: Equivalent to showing the characteristic function $\varphi_{Z_n}(t) \to e^{-t^2/2}$ pointwise, then invoking Lévy's continuity theorem. Equivalently, for every real $z$ ,

4

\quad \lim_{n\to\infty} \mathbb{P}\!\left( \frac{\bar X_n - \mu}{\sigma/\sqrt n} \le z \right) = \Phi(z) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{z} e^{-u^2/2}\, du

Derivation hint: Restate [3] using the CDF definition of convergence in distribution; $\Phi$ is everywhere continuous, so convergence holds at all $z$ .

TheoremLindeberg Condition — the General i.n.i.d. Form

For independent but not necessarily identically distributed $X_k$ with means $\mu_k$ , variances $\sigma_k^2$ , and $s_n^2 = \sum_{k=1}^n \sigma_k^2$ , the standardized sum converges to $N(0,1)$ if the Lindeberg condition holds:

5

\quad \forall \varepsilon > 0:\quad \lim_{n\to\infty} \frac{1}{s_n^2}\sum_{k=1}^n \mathbb{E}\!\left[ (X_k-\mu_k)^2 \,\mathbf{1}_{\{|X_k-\mu_k| > \varepsilon s_n\}} \right] = 0

Derivation hint: The condition guarantees no single summand dominates the total variance; it is the precise statement that each term is "asymptotically negligible."

ProofCharacteristic-Function Proof of the Classical CLT

We prove [3] under the standing assumptions. Write $Y_i = (X_i - \mu)/\sigma$ , so that $\mathbb{E}[Y_i]=0$ , $\operatorname{Var}(Y_i)=1$ , and $Z_n = \frac{1}{\sqrt n}\sum_{i=1}^n Y_i$ .

Step 1 — Reduce to characteristic functions. Let $\varphi(t) = \mathbb{E}[e^{itY_1}]$ be the common characteristic function. By independence,

6

\quad \varphi_{Z_n}(t) = \mathbb{E}\!\left[ \exp\!\left( \tfrac{it}{\sqrt n}\sum_{i=1}^n Y_i \right) \right] = \left[ \varphi\!\left( \tfrac{t}{\sqrt n} \right) \right]^{n}

Derivation hint: The expectation of a product of independent factors equals the product of expectations; each factor equals $\varphi(t/\sqrt n)$ .

Step 2 — Taylor-expand $\varphi$ near 0. Because $\mathbb{E}[Y_1^2]=1<\infty$ , $\varphi$ is twice differentiable at $0$ with $\varphi(0)=1$ , $\varphi'(0)=i\mathbb{E}[Y_1]=0$ , $\varphi''(0)=-\mathbb{E}[Y_1^2]=-1$ . Hence

7

\quad \varphi(u) = 1 - \tfrac{1}{2}u^2 + o(u^2)\quad \text{as } u\to 0

Derivation hint: Differentiate $\varphi(u)=\mathbb{E}[e^{iuY}]$ under the expectation (justified by finite second moment) and evaluate at $u=0$ ; the $o(u^2)$ remainder follows from Taylor's theorem with the Peano form.

Step 3 — Substitute $u = t/\sqrt n$ .

8

\quad \varphi\!\left( \tfrac{t}{\sqrt n} \right) = 1 - \frac{t^2}{2n} + o\!\left( \tfrac{1}{n} \right)

Derivation hint: Replace $u$ by $t/\sqrt n$ in [7]; for fixed $t$ , $u\to 0$ as $n\to\infty$ , so the remainder term is $o(1/n)$ .

Step 4 — Take the $n$ -th power and pass to the limit. Using the standard fact $\big(1 + \frac{a_n}{n}\big)^n \to e^{a}$ when $a_n \to a$ :

9

\quad \varphi_{Z_n}(t) = \left( 1 - \frac{t^2}{2n} + o\!\left(\tfrac1n\right)\right)^{n} \xrightarrow{\ n\to\infty\ } e^{-t^2/2}

Derivation hint: Take complex logarithms — $n\log(1 - \frac{t^2}{2n} + o(1/n)) = -\frac{t^2}{2} + o(1) \to -\frac{t^2}{2}$ — and exponentiate.

Step 5 — Invoke Lévy's continuity theorem. The limit $e^{-t^2/2}$ is the characteristic function of $N(0,1)$ and is continuous at $t=0$ . By Lévy's continuity theorem, pointwise convergence of characteristic functions to a function continuous at the origin implies convergence in distribution. Therefore $Z_n \xrightarrow{d} N(0,1)$ . $\blacksquare$

∎

⚠️ The proof needs only two moments

Notice that Step 2 used only $\mathbb{E}[Y]=0$ and $\mathbb{E}[Y^2]=1$ . No assumption about third moments, skewness, boundedness, or continuity of the distribution was made. This is the mechanical reason for universality — the higher-order Taylor terms scale like $n^{-3/2}$ and vanish faster than the surviving $t^2/2n$ term.

mathematician · Pierre-Simon Laplace | 1749–1827

Though Abraham de Moivre first proved the normal approximation to the binomial in 1733, it was Laplace who, in his Théorie analytique des probabilités (1812), generalized the result far beyond coin tosses and recognized its centrality to the theory of errors. Laplace developed the characteristic-function method in embryonic form and applied the approximation to astronomical and geodetic measurement errors, effectively founding mathematical statistics. The modern rigorous theory — pinning down the exact hypotheses (finite variance, the Lindeberg condition) — was completed only in the early twentieth century by Aleksandr Lyapunov and Jarl Waldemar Lindeberg, with William Feller later proving Lindeberg's condition essentially necessary.

🧠 Intuition

Think of building a tall sandpile by dropping grains. Each grain (a single $X_i$ ) has its own quirky shape and lands wherever it lands, but the height of the pile (the sum) blurs all those individual quirks into a smooth profile. Three intuitions reinforce one another: 1. Convolution smooths. Adding independent variables convolves their densities. Repeated convolution is a smoothing operation — sharp corners and spikes get rounded off, and the rounded-off limit is the smoothest "spread" with a fixed variance, which is the Gaussian. 2. Two-moment bottleneck. Rescaling by $\sqrt n$ keeps the variance fixed at 1 while squeezing all higher cumulants toward zero. The Gaussian is the only distribution whose cumulants beyond the second are all zero — so it is the natural attractor. 3. Entropy maximization. Among all distributions with a given mean and variance, the Gaussian has maximum entropy. Each averaging step can only add entropy (information about the original shape is lost), so the limit is the maximum-entropy shape compatible with the fixed variance. The deep moral: the bell curve appears not because nature "prefers" it, but because it is the unavoidable shape that emerges when many small, independent, comparable influences add up.

Figure

Convergence of the standardized sum of i.i.d. uniform variables toward the standard normal density. Histograms for n=1, 2, 5 are progressively closer to the Gaussian curve.

ExampleSample Mean of Die Rolls

Given. A fair six-sided die is rolled $n=100$ times. Each roll $X_i$ is uniform on $\{1,2,3,4,5,6\}$ . Find the approximate probability that the sample mean $\bar X_{100}$ exceeds $3.7$ .

Step 1 — Moments of one roll.

\mu = \frac{1+2+3+4+5+6}{6} = 3.5, \qquad \sigma^2 = \frac{1}{6}\sum_{k=1}^6 (k-3.5)^2 = \frac{35}{12} \approx 2.9167.

So $\sigma \approx 1.7078$ .

Step 2 — Standard error of the mean.

\frac{\sigma}{\sqrt n} = \frac{1.7078}{\sqrt{100}} = 0.17078.

Step 3 — Standardize the target.

z = \frac{3.7 - 3.5}{0.17078} = \frac{0.2}{0.17078} \approx 1.171.

Step 4 — Apply the CLT.

\mathbb{P}(\bar X_{100} > 3.7) \approx 1 - \Phi(1.171) \approx 1 - 0.8792 = 0.1208.

Answer.

\boxed{\ \mathbb{P}(\bar X_{100} > 3.7) \approx 0.121\ } \quad \text{(dimensionless probability)}

ExampleTotal Weight with a Continuity Correction

Given. A shipment contains $n = 64$ independent boxes. Each box weight $X_i$ has mean $\mu = 5\text{ kg}$ and standard deviation $\sigma = 0.8\text{ kg}$ (distribution unspecified). A truck has a load limit of $325\text{ kg}$ . Estimate the probability the total weight $S_{64}$ exceeds the limit.

Step 1 — Moments of the total.

\mathbb{E}[S_{64}] = n\mu = 64 \times 5 = 320\text{ kg}, \qquad \operatorname{Var}(S_{64}) = n\sigma^2 = 64 \times 0.64 = 40.96\text{ kg}^2.

So $\mathrm{SD}(S_{64}) = \sqrt{40.96} = 6.4\text{ kg}$ .

Step 2 — Standardize the threshold.

z = \frac{325 - 320}{6.4} = \frac{5}{6.4} \approx 0.781.

Step 3 — Apply the CLT.

\mathbb{P}(S_{64} > 325) \approx 1 - \Phi(0.781) \approx 1 - 0.7826 = 0.2174.

Answer.

\boxed{\ \mathbb{P}(S_{64} > 325\text{ kg}) \approx 0.217\ }

(Note: no continuity correction is applied here because the box weights are continuous. For a sum of discrete integer-valued variables one would replace $325$ by $325.5$ — the half-integer continuity correction.)

⚠️ Five things the CLT does NOT say

1. It is not about a single sample becoming normal. The CLT concerns the sampling distribution of the mean/sum, not the distribution of the individual data values. A skewed population stays skewed no matter how large $n$ is. 2. Large $n$ alone is not enough. Heavily skewed or heavy-tailed populations may need $n$ in the hundreds or thousands before the approximation is good; the common " $n \ge 30$ " rule of thumb is a crude heuristic, not a theorem. 3. Finite variance is essential. Without it, the classical CLT fails (see the Cauchy counterexample). 4. Independence (or weak dependence) matters. Strongly correlated data — e.g., a time series with long-range dependence — need not obey the standard CLT. 5. The approximation is worst in the tails. The relative error of $\Phi$ for rare events is governed by Large Deviations Theory, not the CLT, which controls the center of the distribution.

CounterexampleThe Cauchy distribution — infinite variance defeats the CLT

Let $X_1, \dots, X_n$ be i.i.d. standard Cauchy, with density $f(x) = \frac{1}{\pi(1+x^2)}$ . The mean is undefined and the variance is infinite, so the CLT hypotheses fail. Remarkably, the sample mean $\bar X_n$ is itself standard Cauchy for every $n$ :

10

\quad \varphi_{X}(t) = e^{-|t|} \;\Rightarrow\; \varphi_{\bar X_n}(t) = \left[\varphi_X(t/n)\right]^n = e^{-n|t|/n} = e^{-|t|}

Derivation hint: The Cauchy characteristic function $e^{-|t|}$ is stable — raising it to the $n$ -th power at argument $t/n$ reproduces it exactly, so averaging does nothing to tame the spread. Averaging never concentrates the Cauchy distribution: $\bar X_{10^6}$ is exactly as spread out as $X_1$ . There is no Gaussian limit. This is the canonical demonstration that finite variance is not a technical luxury but a genuine requirement, and it foreshadows the broader theory of Stable Distributions.

connection · Where the CLT sits in probability theory

The CLT is one pillar of the classical limit theorems, complementing the Law of Large Numbers: the LLN says $\bar X_n \to \mu$ (the average converges to a constant), while the CLT describes the $O(1/\sqrt n)$ fluctuations around that constant. The two together form a "law plus error bar." The CLT generalizes in several directions: the Berry–Esseen Theorem supplies an explicit rate of convergence $|F_{Z_n}(x) - \Phi(x)| \le C\rho/(\sigma^3\sqrt n)$ ; Stable Distributions replace the Gaussian as the attractor when variance is infinite; Large Deviations Theory governs the exponentially small tail probabilities the CLT cannot resolve; and the Martingale Central Limit Theorem extends the result to dependent sequences. The Gaussian limit itself is the universal object studied in The Normal Distribution, and the proof technique relies entirely on the Characteristic Function and Lévy's continuity theorem.

🔬 Concrete uses across the sciences

⏳ History

- 1733 — Abraham de Moivre proves the first version: the normal approximation to the binomial distribution (the de Moivre–Laplace theorem), motivated by gambling problems, in a privately circulated note later included in The Doctrine of Chances.

- 1810–1812 — Pierre-Simon Laplace generalizes the result well beyond the binomial in his Théorie analytique des probabilités, introducing generating-function (characteristic-function) techniques and applying the approximation to the theory of measurement errors.

- 1809 — Carl Friedrich Gauss independently derives the normal "error curve" from the method of least squares, cementing the link between the Gaussian and aggregated random error.

- 1853–1860s — Pafnuty Chebyshev initiates the rigorous Russian school, developing the method of moments to attack limit theorems.

- 1901 — Aleksandr Lyapunov gives the first fully rigorous general proof using characteristic functions, under his Lyapunov condition (a moment condition slightly stronger than Lindeberg's).

- 1920 — George Pólya coins the name "zentraler Grenzwertsatz" (central limit theorem), where "central" denotes its centrality to probability theory.

- 1922 — Jarl Waldemar Lindeberg establishes the optimal Lindeberg condition for the i.n.i.d. case via an elegant elementary swapping argument.

- 1935 — William Feller (and independently Paul Lévy) prove that the Lindeberg condition is essentially necessary as well as sufficient (the Lindeberg–Feller theorem), closing the classical question.

- 1941–1956 — Berry and Esseen independently establish the first explicit, uniform bound on the rate of convergence, quantifying how large

n

must be.

open-problem · Beyond the classical hypotheses

The classical CLT is closed as a theorem, but its frontier is alive. Heavy tails: when $\operatorname{Var}=\infty$ but the tails decay as a power law $\mathbb{P}(|X|>x)\sim x^{-\alpha}$ with $0<\alpha<2$ , the correct attractor is an $\alpha$ -stable law, not the Gaussian — a complete generalized CLT exists but identifying the exact domain of attraction of a given distribution can be delicate. Dependence: for stationary sequences, a CLT holds under mixing or martingale-difference conditions, but quantifying the effective sample size under long-range dependence (where the variance of $S_n$ grows faster than $n$ ) remains an active research area with anomalous $n^{H}$ scaling, $H \neq 1/2$ . High dimensions: modern statistics needs CLTs for vectors and random matrices where the dimension $d$ grows with $n$ ; obtaining sharp, dimension-free rates (a program advanced by Chernozhukov–Chetverikov–Kato and others) is current research. Quantitative optimality: the best universal constant in the Berry–Esseen bound is still not exactly known — the current best upper bound is roughly $C \le 0.4748$ , while the conjectured optimal constant is smaller.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also