Central Limit Theorem

⚡ Key Takeaways

One-line statement: The sum (or average) of a large number of independent, identically distributed random variables, suitably standardized, converges in distribution to a normal (Gaussian) distribution — regardless of the original distribution's shape (provided finite variance).

Key result in words: Aggregated randomness becomes Gaussian. Universal "bell curve" behavior emerges from almost any underlying randomness.

Where it applies: Statistical inference, confidence intervals, hypothesis testing, error analysis, measurement, finance, statistical physics, signal processing.

Who/when: De Moivre (1733, binomial case), Laplace (1810), and rigorously generalized by Lyapunov (1901) and Lindeberg–Lévy (1920s).

💡 The Core Insight

Independent random fluctuations, when summed and rescaled by $\sqrt{n}$ , lose all memory of their individual distributions. The normal distribution is a fixed point (attractor) of the operation "add two independent copies and rescale." Because the Gaussian is the unique distribution (with finite variance) that is stable under this convolution-and-rescaling operation, repeated aggregation drags everything toward it. The $\sqrt{n}$ scaling is not arbitrary: it is precisely the rate at which the standard deviation of a sum of $n$ independent terms grows, so it is the unique normalization that yields a nondegenerate limit.

DefinitionConvergence in Distribution

A sequence of random variables $Y_n$ converges in distribution to $Y$ (written $Y_n \xrightarrow{d} Y$ ) if their cumulative distribution functions converge at every continuity point of $F_Y$ :

1

\quad \lim_{n\to\infty} F_{Y_n}(x) = F_Y(x) \quad \text{for all } x \text{ where } F_Y \text{ is continuous.}

Derivation hint: This is the weakest standard mode of stochastic convergence; pointwise CDF convergence is required only where the limit CDF is continuous, which for $N(0,1)$ is everywhere. Here $F_{Y_n}(x) = \Pr(Y_n \le x)$ . Symbols: $Y_n$ = standardized partial sum; $Y$ = limiting standard normal.

DefinitionStandardized Sum and Sample Mean

Let $X_1,\dots,X_n$ be i.i.d. with $\mathbb{E}[X_i]=\mu$ and $\operatorname{Var}(X_i)=\sigma^2 \in (0,\infty)$ . Define $S_n = \sum_{i=1}^n X_i$ and $\bar X_n = S_n/n$ . The standardized variable is

2

\quad Z_n = \frac{S_n - n\mu}{\sigma\sqrt{n}} = \frac{\bar X_n - \mu}{\sigma/\sqrt{n}}.

Derivation hint: Subtract the mean $\mathbb{E}[S_n]=n\mu$ and divide by the standard deviation $\sqrt{\operatorname{Var}(S_n)} = \sigma\sqrt{n}$ since variances of independent terms add. Symbols: $S_n$ = sum; $\bar X_n$ = sample mean; $\mu$ = population mean; $\sigma$ = population standard deviation; $n$ = sample size.

TheoremClassical (Lindeberg–Lévy) Central Limit Theorem

Let $X_1, X_2, \dots$ be i.i.d. with finite mean $\mu$ and finite, nonzero variance $\sigma^2$ . Then

3

\quad Z_n = \frac{S_n - n\mu}{\sigma\sqrt{n}} \xrightarrow{\,d\,} N(0,1),

Derivation hint: Apply the characteristic-function / Lévy continuity argument below; equivalently, the limiting CDF equals $\Phi(x)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^x e^{-t^2/2}\,dt$ . Equivalently, $\Pr\!\left(\frac{S_n-n\mu}{\sigma\sqrt n}\le x\right)\to \Phi(x)$ for every real $x$ .

TheoremLyapunov CLT (non-identical case)

Let $X_1,\dots,X_n$ be independent (not necessarily identical) with means $\mu_i$ , variances $\sigma_i^2$ , and $s_n^2 = \sum_{i=1}^n \sigma_i^2$ . If for some $\delta>0$ the Lyapunov condition holds,

4

\quad \lim_{n\to\infty}\frac{1}{s_n^{2+\delta}}\sum_{i=1}^n \mathbb{E}\big[\,|X_i-\mu_i|^{2+\delta}\big] = 0,

Derivation hint: This controls the contribution of any single term to the total variance, preventing one variable from dominating; it implies the Lindeberg condition. then $\frac{1}{s_n}\sum_{i=1}^n (X_i-\mu_i) \xrightarrow{d} N(0,1)$ .

TheoremLindeberg CLT — sufficient & (essentially) necessary

Under the same independent (non-identical) setup, if the Lindeberg condition holds for every $\varepsilon>0$ ,

5

\quad \lim_{n\to\infty}\frac{1}{s_n^2}\sum_{i=1}^n \mathbb{E}\big[(X_i-\mu_i)^2\,\mathbf{1}_{\{|X_i-\mu_i|>\varepsilon s_n\}}\big]=0,

Derivation hint: The indicator truncates to "large deviations"; the condition demands that no large-deviation mass survives relative to total variance. then $\frac{1}{s_n}\sum (X_i-\mu_i)\xrightarrow{d} N(0,1)$ . Conversely (Feller–Lévy), under the negligibility condition $\max_i \sigma_i^2/s_n^2 \to 0$ , Lindeberg's condition is also necessary.

ProofCharacteristic Function Proof of the Classical CLT

We use the characteristic function $\varphi_X(t)=\mathbb{E}[e^{itX}]$ and Lévy's continuity theorem: pointwise convergence of characteristic functions to a function continuous at $0$ implies convergence in distribution.

Step 1 — Reduce to mean 0, variance 1. Let $W_i = (X_i-\mu)/\sigma$ , so $\mathbb{E}[W_i]=0$ , $\operatorname{Var}(W_i)=1$ . Then $Z_n = \frac{1}{\sqrt n}\sum_{i=1}^n W_i$ .

Step 2 — Characteristic function of the sum. By independence,

6

\quad \varphi_{Z_n}(t) = \mathbb{E}\!\left[\exp\!\left(\frac{it}{\sqrt n}\sum_{i=1}^n W_i\right)\right] = \left[\varphi_W\!\left(\frac{t}{\sqrt n}\right)\right]^n.

Derivation hint: The characteristic function of a sum of independent variables is the product of their characteristic functions; identical distribution gives the $n$ -th power.

Step 3 — Taylor expand near 0. Since $\mathbb{E}[W]=0$ and $\mathbb{E}[W^2]=1$ , a second-order Taylor expansion gives

7

\quad \varphi_W(u) = 1 - \tfrac{1}{2}u^2 + o(u^2)\quad\text{as } u\to 0.

Derivation hint: Differentiate under the expectation: $\varphi_W'(0)=i\mathbb{E}[W]=0$ , $\varphi_W''(0)=-\mathbb{E}[W^2]=-1$ ; finite variance guarantees the remainder is $o(u^2)$ .

Step 4 — Substitute and take the limit. With $u=t/\sqrt n$ ,

8

\quad \varphi_{Z_n}(t) = \left(1 - \frac{t^2}{2n} + o\!\left(\frac{1}{n}\right)\right)^n \xrightarrow[n\to\infty]{} e^{-t^2/2}.

Derivation hint: Use $\big(1+\frac{a_n}{n}\big)^n \to e^a$ when $a_n\to a$ ; here $a_n \to -t^2/2$ .

Step 5 — Identify the limit. The function $e^{-t^2/2}$ is exactly the characteristic function of $N(0,1)$ . Since it is continuous at $t=0$ , Lévy's continuity theorem yields $Z_n \xrightarrow{d} N(0,1)$ . $\blacksquare$

∎

PL

Pierre-Simon Laplace

1749–1827

Laplace gave the first general analytic treatment of the CLT in his Théorie analytique des probabilités (1812), extending De Moivre's binomial result to broad classes of averages using what we would now recognize as characteristic-function (Fourier) techniques. He grasped that error distributions of measurements tend to the normal law and applied this to celestial mechanics and the combination of astronomical observations. His "central limit" insight underpinned the method of least squares (developed alongside Gauss and Legendre) and made the normal distribution the canonical model of measurement error.

🧠 Intuition

Picture rolling a single fair die: the distribution is flat (uniform on 1–6). Roll two dice and sum: you get a triangular peak at 7. Roll ten dice and sum: the histogram is already strikingly bell-shaped. Each additional independent contribution averages out extremes — to be far from the mean, many independent terms must conspire to all be large at once, which is exponentially unlikely. The Gaussian shape is the "maximally noncommittal" distribution consistent with a fixed mean and variance (it maximizes entropy), so when summing erases all other information, what remains is the maximally-spread bell curve. The $\sqrt n$ scaling reflects that the sum's spread grows like $\sqrt n$ while its mean grows like $n$ : the relative fluctuation $\propto 1/\sqrt n$ shrinks, which is why averages stabilize.

Figure

Convergence of the standardized sum of uniform random variables to the normal curve as n grows

ExampleSample Mean of Service Times

Given: A help desk's call-handling time has mean $\mu = 4$ minutes and standard deviation $\sigma = 3$ minutes (distribution is right-skewed, not normal). A manager averages $n=100$ independent calls. What is the approximate probability the sample mean exceeds 4.5 minutes?

Step 1 — Standard error of the mean. $\operatorname{SE} = \sigma/\sqrt n = 3/\sqrt{100} = 0.3$ min.

Step 2 — Invoke the CLT. By Equation [3], $\bar X_{100} \approx N(\mu, \operatorname{SE}^2) = N(4, 0.09)$ even though individual times are skewed.

Step 3 — Standardize the threshold.

9

\quad z = \frac{4.5 - 4}{0.3} = 1.667.

Derivation hint: Apply Equation [2] with $\bar X_n = 4.5$ , $\mu=4$ , $\sigma/\sqrt n = 0.3$ .

Step 4 — Read the normal tail. $\Pr(Z>1.667) = 1 - \Phi(1.667) \approx 1 - 0.9522 = 0.0478$ .

\boxed{\Pr(\bar X_{100} > 4.5\text{ min}) \approx 0.048 \;(\text{about } 4.8\%)}

ExampleTotal of Many Rounding Errors

Given: $n = 1000$ numbers are each rounded to the nearest integer. Each rounding error is i.i.d. Uniform on $[-0.5, 0.5]$ , so $\mu=0$ and $\sigma^2 = \frac{(1)^2}{12} = \frac{1}{12}$ . Estimate $\Pr(|S_n| > 10)$ for the accumulated error $S_n=\sum_{i=1}^{1000} \varepsilon_i$ .

Step 1 — Mean and variance of the sum. $\mathbb{E}[S_n]=0$ ; $\operatorname{Var}(S_n)= n\sigma^2 = 1000/12 = 83.33$ , so $\operatorname{SD}(S_n)=\sqrt{83.33}=9.129$ .

Step 2 — CLT approximation. $S_n \approx N(0, 83.33)$ by Equation [3].

Step 3 — Standardize.

10

\quad z = \frac{10 - 0}{9.129} = 1.0954.

Derivation hint: Use $S_n - n\mu = 10$ over $\sigma\sqrt n = \sqrt{1000/12}$ .

Step 4 — Two-tailed probability. $\Pr(|S_n|>10) = 2\big(1-\Phi(1.0954)\big) \approx 2(1-0.8633) = 0.2734$ .

\boxed{\Pr(|S_n| > 10) \approx 0.27 \;(\text{about } 27\%)}

⚠️ Common mistake

The CLT is NOT about the data becoming normal. It is about the distribution of the sample mean (or sum) becoming normal as $n$ grows. The raw data keep their original (possibly skewed) distribution no matter how large $n$ is. Two further pitfalls: (1) " $n>30$ is always enough" is folklore — heavily skewed or heavy-tailed distributions may require $n$ in the thousands; (2) the classical CLT requires finite variance. Distributions with infinite variance (e.g., Cauchy, or Pareto with tail index $\le 2$ ) do not obey the classical CLT; their normalized sums converge instead to non-Gaussian stable distributions.

CounterexampleCauchy Distribution: No CLT

If $X_1,\dots,X_n$ are i.i.d. standard Cauchy (density $\frac{1}{\pi(1+x^2)}$ ), the mean and variance do not exist. Remarkably, the sample mean $\bar X_n$ has exactly the same Cauchy distribution for every $n$ — averaging brings no concentration at all:

11

\quad \bar X_n = \frac1n\sum_{i=1}^n X_i \;\stackrel{d}{=}\; X_1 \quad\text{(Cauchy, for all } n).

Derivation hint: The Cauchy characteristic function is $\varphi(t)=e^{-|t|}$ ; then $\varphi_{\bar X_n}(t)=\varphi(t/n)^n = e^{-n|t|/n}=e^{-|t|}$ , unchanged. This shows finite variance is essential: the standardized sum does not converge to a normal law.

connection

The CLT sits at the heart of probability and statistics. It refines the Law of Large Numbers, which says $\bar X_n \to \mu$ (a first-order statement); the CLT gives the second-order fluctuation $\bar X_n - \mu \approx N(0,\sigma^2/n)$ . It is proved using the Characteristic Function and Lévy's Continuity Theorem, and is the mechanism behind the Normal Distribution's dominance. Its precision is quantified by the Berry–Esseen Theorem (convergence rate $O(1/\sqrt n)$ ) and improved by Edgeworth Expansion (skewness/kurtosis corrections). Generalizations include the Multivariate Central Limit Theorem, Martingale Central Limit Theorem, and the theory of Stable Distributions for the infinite-variance case. In statistics it underlies the Student's t-distribution, Confidence Interval construction, and Hypothesis Testing.

🔬 Where this appears

⏳ History

- 1733 — Abraham de Moivre: Derives the normal approximation to the binomial distribution (the De Moivre–Laplace theorem) for coin-tossing — the first special case of the CLT.

- 1810–1812 — Pierre-Simon Laplace: Provides a general analytic statement using Fourier/characteristic-function methods in Théorie analytique des probabilités, applying it to measurement errors and astronomy.

- 1809–1823 — Carl Friedrich Gauss: Develops the normal error law and least squares, intertwining the Gaussian with the theory of observational error.

- 1887–1898 — Pafnuty Chebyshev & Andrey Markov: Begin rigorous treatment via the method of moments, identifying the need for careful hypotheses.

- 1901 — Aleksandr Lyapunov: Gives the first rigorous general proof under the Lyapunov condition (Equation [4]), using characteristic functions, freeing the result from the i.i.d. assumption.

- 1920–1922 — Jarl Waldemar Lindeberg: Establishes the sharp Lindeberg condition (Equation [5]) and a now-classic "replacement" proof.

- 1922 — George Pólya: Coins the name zentraler Grenzwertsatz ("central limit theorem"), where "central" denotes its central importance in probability theory.

- 1935 — Paul Lévy & William Feller: Prove that the Lindeberg condition is essentially necessary (under uniform negligibility), completing the classical theory.

- 1941–1942 — Andrew Berry & Carl-Gustav Esseen: Quantify the rate of convergence, giving the

O(1/\sqrt n)

uniform bound on the CDF approximation.

open-problem

Beyond i.i.d. and finite variance. Several frontiers refine or extend the CLT. (1) Rates and dependence: sharp Berry–Esseen constants are still being optimized, and CLTs for strongly dependent or long-memory processes (e.g., fractional Gaussian noise) require entirely different normalizations and can yield non-Gaussian limits (Rosenblatt distribution). (2) High dimensions: modern statistics needs CLTs where dimension $d$ grows with $n$ (Chernozhukov–Chetverikov–Kato bounds for maxima over high-dimensional sums) — an active research area. (3) Heavy tails: when variance is infinite, normalized sums converge to $\alpha$ -stable laws; characterizing domains of attraction and convergence rates remains subtle. (4) Concentration and finite-sample validity: how large must $n$ be for a given accuracy and a given skewness/kurtosis? Edgeworth and saddlepoint expansions help, but practical, distribution-aware guarantees are an ongoing practical challenge. (5) Free probability: the free CLT gives the semicircle law as the analog for non-commuting (random matrix) variables — a deep structural parallel.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also