The Gaussian Distribution

⚡ Key Takeaways

The Gaussian distribution (or normal distribution) is the continuous probability distribution with density $f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}$ , the bell-shaped curve parameterised by mean $\mu$ and standard deviation $\sigma$ . It is the universal limiting distribution of sums of many independent random influences (the Central Limit Theorem), making it ubiquitous in measurement error, statistics, and the natural sciences. Introduced via error theory by Gauss (1809) and Laplace, building on de Moivre (1733).

💡 The Core Insight

A single bell curve, fully pinned down by just two numbers — where it's centred ( $\mu$ ) and how wide it is ( $\sigma$ ) — is the inevitable shape that emerges whenever many small, independent random effects pile up additively. Everything else about the Gaussian (its tails, its $68$ – $95$ – $99.7$ rule, its self-reproducing behaviour) flows from this single algebraic form $e^{-x^2}$ .

DefinitionGaussian / Normal Distribution

A real-valued random variable $X$ is Gaussian (normally distributed) with mean $\mu \in \mathbb{R}$ and variance $\sigma^2 > 0$ , written $X \sim \mathcal{N}(\mu, \sigma^2)$ , if its probability density function is

1

\quad f(x) = \frac{1}{\sigma\sqrt{2\pi}}\,\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right), \qquad x \in \mathbb{R}.

Derivation hint: This is the unique form (up to normalisation) satisfying $f'(x)/f(x) = -(x-\mu)/\sigma^2$ , the differential equation Gauss derived from requiring the sample mean to be the maximum-likelihood estimate of the location. Here $\mu$ is the mean (location, the point of symmetry), $\sigma > 0$ is the standard deviation (scale), and $\sigma^2$ is the variance. The prefactor $\frac{1}{\sigma\sqrt{2\pi}}$ is the normalising constant ensuring the total probability is $1$ . The standard normal is the special case $\mu=0$ , $\sigma=1$ , with density conventionally denoted

2

\quad \varphi(z) = \frac{1}{\sqrt{2\pi}}\,e^{-z^2/2}, \qquad \Phi(z)=\int_{-\infty}^{z}\varphi(t)\,dt.

Derivation hint: Substitute $z=(x-\mu)/\sigma$ into [1]; the Jacobian $dx=\sigma\,dz$ cancels the $1/\sigma$ factor. Any Gaussian relates to the standard one by the affine standardisation $Z = (X-\mu)/\sigma \sim \mathcal{N}(0,1)$ .

TheoremNormalisation: the Gaussian Integral

The density [1] integrates to $1$ :

3

\quad \int_{-\infty}^{\infty} \frac{1}{\sigma\sqrt{2\pi}}\,e^{-(x-\mu)^2/2\sigma^2}\,dx = 1, \quad\text{equivalently}\quad \int_{-\infty}^{\infty} e^{-t^2}\,dt = \sqrt{\pi}.

Derivation hint: Square the integral, convert the resulting double integral to polar coordinates, where $e^{-r^2}r\,dr$ integrates elementarily.

TheoremMoments

If $X\sim\mathcal{N}(\mu,\sigma^2)$ then

4

\quad \mathbb{E}[X]=\mu,\qquad \operatorname{Var}(X)=\sigma^2,\qquad \mathbb{E}\big[(X-\mu)^{2k}\big]=\sigma^{2k}\,(2k-1)!!,

and all odd central moments vanish. The moment-generating function and characteristic function are

5

\quad M_X(t)=\mathbb{E}[e^{tX}]=e^{\mu t + \tfrac12\sigma^2 t^2},\qquad \phi_X(t)=\mathbb{E}[e^{itX}]=e^{i\mu t - \tfrac12\sigma^2 t^2}.

Derivation hint: Complete the square in the exponent $tx-(x-\mu)^2/2\sigma^2$ ; the remaining Gaussian integrates to $1$ , leaving the displayed exponential.

ProofThe Gaussian Integral, Equation 3

We prove $I=\int_{-\infty}^{\infty}e^{-t^2}\,dt=\sqrt{\pi}$ ; the general case [3] follows by the substitution $t=(x-\mu)/(\sigma\sqrt2)$ .

Step 1 — Square the integral. Because $I$ is a constant,

I^2 = \left(\int_{-\infty}^{\infty}e^{-x^2}\,dx\right)\left(\int_{-\infty}^{\infty}e^{-y^2}\,dy\right)=\iint_{\mathbb{R}^2} e^{-(x^2+y^2)}\,dx\,dy.

Step 2 — Polar coordinates. Set $x=r\cos\theta$ , $y=r\sin\theta$ , so $x^2+y^2=r^2$ and $dx\,dy=r\,dr\,d\theta$ :

I^2 = \int_{0}^{2\pi}\!\!\int_{0}^{\infty} e^{-r^2}\,r\,dr\,d\theta.

Step 3 — Evaluate. The inner integral uses $u=r^2$ , $du=2r\,dr$ :

\int_0^\infty e^{-r^2}r\,dr = \tfrac12\int_0^\infty e^{-u}\,du = \tfrac12.

Hence $I^2 = 2\pi\cdot\tfrac12 = \pi$ , so $I=\sqrt{\pi}$ .

Step 4 — Recover [3]. With $t=(x-\mu)/(\sigma\sqrt2)$ , $dx=\sigma\sqrt2\,dt$ :

\int_{-\infty}^{\infty}\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}\,dx = \frac{\sigma\sqrt2}{\sigma\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-t^2}\,dt = \frac{1}{\sqrt{\pi}}\cdot\sqrt{\pi}=1. \qquad\blacksquare

∎

ProofSums of Independent Gaussians Are Gaussian

Let $X_1\sim\mathcal{N}(\mu_1,\sigma_1^2)$ and $X_2\sim\mathcal{N}(\mu_2,\sigma_2^2)$ be independent.

Step 1. The characteristic function of a sum of independents is the product: $\phi_{X_1+X_2}(t)=\phi_{X_1}(t)\,\phi_{X_2}(t)$ .

Step 2. By [5],

\phi_{X_1+X_2}(t)=e^{i\mu_1 t-\tfrac12\sigma_1^2 t^2}\cdot e^{i\mu_2 t-\tfrac12\sigma_2^2 t^2}=e^{i(\mu_1+\mu_2)t-\tfrac12(\sigma_1^2+\sigma_2^2)t^2}.

Step 3. This is exactly the characteristic function of $\mathcal{N}(\mu_1+\mu_2,\ \sigma_1^2+\sigma_2^2)$ . By the uniqueness of characteristic functions,

6

\quad X_1+X_2 \sim \mathcal{N}\big(\mu_1+\mu_2,\ \sigma_1^2+\sigma_2^2\big). \qquad\blacksquare

∎

mathematician · Carl Friedrich Gauss | 1777–1855

Gauss arrived at the distribution in his 1809 Theoria Motus Corporum Coelestium while tackling the practical problem of fitting orbits to noisy astronomical observations (he had famously recovered the asteroid Ceres in 1801). His argument was a piece of inverse reasoning: he postulated that the most probable value of a measured quantity should be the arithmetic mean of the observations, then asked which error law forces this. The answer was the bell curve, and the method of least squares fell out naturally. Pierre-Simon Laplace soon connected it to his earlier limit theorems, giving the distribution its deeper theoretical standing through what became the Central Limit Theorem. The curve was therefore not invented in a vacuum but reverse-engineered from a demand about estimators — a rare instance of a fundamental object being defined by the property it must produce.

🧠 Intuition

Think of a Galton board (a "bean machine"): a ball drops through rows of pegs, bouncing left or right at each one. Each bounce is a tiny independent random step. After many rows, the ball's horizontal position is a sum of many independent $\pm1$ steps, and the heap of balls at the bottom traces out a bell curve. This is the Central Limit Theorem made physical — no single peg matters; only the accumulation does. The shape $e^{-x^2}$ encodes a competition. Near the centre the curve is nearly flat (the mean is overwhelmingly likely), but the exponent grows quadratically, so as you move out the density is crushed faster and faster — the tails are extraordinarily thin. A value $4\sigma$ from the mean is about $16{,}000$ times less likely than the mean itself. The two parameters do orthogonal jobs: $\mu$ slides the whole curve left or right without changing its shape, while $\sigma$ stretches it horizontally (and compresses it vertically to keep area $1$ ). The famous rule of thumb captures the spread: roughly $68\%$ of probability lies within $1\sigma$ , $95\%$ within $2\sigma$ , and $99.7\%$ within $3\sigma$ .

Figure

The standard normal density φ(z) with the 68–95–99.7 regions

ExampleIQ Scores and a Threshold Probability

Given: IQ scores are modelled as $X\sim\mathcal{N}(\mu=100,\ \sigma^2=15^2)$ . Find $P(X > 130)$ .

Step 1 — Standardise. Convert to a $z$ -score using $Z=(X-\mu)/\sigma$ :

z = \frac{130-100}{15} = \frac{30}{15} = 2.00.

Step 2 — Use the standard normal CDF. We need $P(Z>2.00)=1-\Phi(2.00)$ . From tables, $\Phi(2.00)=0.9772$ .

Step 3 — Compute.

P(X>130)=1-0.9772 = 0.0228.

Step 4 — Interpret. About $2.28\%$ of the population scores above $130$ — consistent with the $95\%$ -within- $2\sigma$ rule (the two tails together hold $\approx 5\%$ , so one tail holds $\approx 2.5\%$ ).

\boxed{P(X>130)\approx 0.0228 \ \ (2.28\%)}

ExampleAveraging Reduces Spread — the √n Law

Given: A machined part has length $X\sim\mathcal{N}(\mu=50.0\text{ mm},\ \sigma=0.8\text{ mm})$ . A quality engineer averages $n=16$ independent parts. Find the distribution of the sample mean $\bar X$ and the probability $P(\bar X > 50.4\text{ mm})$ .

Step 1 — Distribution of the mean. By [6] applied repeatedly, the sum of $16$ independent Gaussians is Gaussian, and dividing by $16$ scales mean by $1$ and standard deviation by $1/16\times\sqrt{16}$ :

\bar X \sim \mathcal{N}\!\left(\mu,\ \frac{\sigma^2}{n}\right)=\mathcal{N}\!\left(50.0,\ \frac{0.8^2}{16}\right).

Step 2 — Standard error.

\sigma_{\bar X} = \frac{\sigma}{\sqrt n} = \frac{0.8}{\sqrt{16}} = \frac{0.8}{4} = 0.20\text{ mm}.

Step 3 — Standardise the threshold.

z = \frac{50.4-50.0}{0.20} = \frac{0.4}{0.20} = 2.00.

Step 4 — Probability.

P(\bar X > 50.4) = 1-\Phi(2.00) = 1-0.9772 = 0.0228.

Interpretation: A single part exceeds $50.4$ mm with probability $P(Z>0.5)=0.309$ , but the average of $16$ exceeds it with probability only $0.0228$ — averaging shrinks fluctuations by the factor $\sqrt n=4$ .

\boxed{\sigma_{\bar X}=0.20\text{ mm},\qquad P(\bar X>50.4)\approx 0.0228}

⚠️ Common mistake

"Everything is normally distributed." False. Many real quantities are skewed (incomes, lifetimes, file sizes), bounded (proportions), or heavy-tailed (financial returns, city sizes). The Gaussian is the default only when a quantity is the additive sum of many small independent effects. Multiplicative effects produce the log-normal, not the normal, distribution.

"Zero correlation implies independence." False in general. It is true only for jointly Gaussian variables. Two variables can be uncorrelated yet strongly dependent (see the counterexample).

"Uncorrelated marginal Gaussians are jointly Gaussian." False. Two random variables can each be marginally $\mathcal{N}(0,1)$ without the pair $(X,Y)$ being bivariate normal.

" $\sigma$ and $\sigma^2$ are interchangeable." They are not — variances add under independence, standard deviations do not. Always check whether a formula calls for $\sigma$ (a length scale, units of $X$ ) or $\sigma^2$ (units of $X^2$ ).

CounterexampleUncorrelated but Dependent

Let $X\sim\mathcal{N}(0,1)$ and define $Y=X^2$ . Then $Y$ is a deterministic function of $X$ (maximal dependence), yet

\operatorname{Cov}(X,Y)=\mathbb{E}[X^3]-\mathbb{E}[X]\mathbb{E}[X^2]=0-0\cdot 1 = 0,

since all odd moments of a centred Gaussian vanish (by [4]). So $X$ and $Y$ are uncorrelated but not independent — showing that the convenient "zero correlation ⇒ independence" rule is exclusive to the jointly Gaussian setting and fails otherwise.

connection

The Gaussian sits at the centre of a web of probabilistic structures. The Central Limit Theorem explains why it is universal: sums of independent, finite-variance random variables converge to it. Squaring and summing independent standard normals produces the Chi-Squared Distribution, which underlies variance estimation and goodness-of-fit tests; ratios involving these give the Student's t-Distribution and the F-Distribution. As the continuous limit of the Binomial Distribution (the de Moivre–Laplace theorem), it links discrete and continuous probability. Its self-reproducing behaviour under convolution marks it as the prototypical member of the Stable Distribution family, and its role as the maximum-entropy distribution for fixed mean and variance connects it to Information Theory. Finally, the Gaussian is the fundamental solution (heat kernel) of the Heat Equation, tying it to diffusion and Brownian motion.

🔬 Where this appears

⏳ History

- 1733 — Abraham de Moivre derives the bell curve as the limiting approximation to the binomial distribution for large

n

, in pursuit of approximating

\binom{n}{k}

in games of chance — the first appearance of the form

e^{-x^2}

.

- 1774 — Pierre-Simon Laplace investigates error distributions and the distribution of the arithmetic mean, developing early limit theorems.

- 1809 — Carl Friedrich Gauss publishes Theoria Motus, deriving the normal law from the postulate that the sample mean is the most probable value, and founding the method of least squares.

- 1810 — Laplace proves a general form of the Central Limit Theorem, establishing the distribution's deep universality and connecting it to Gauss's error law.

- 1823 — Gauss publishes Theoria Combinationis, generalising least squares (the Gauss–Markov theorem) without assuming normality of errors.

- 1875 — Francis Galton introduces the "quincunx" (Galton board) and applies the normal curve to biological variation, coining ideas of regression toward the mean.

- 1894 — Karl Pearson popularises the term "normal distribution" and develops the method of moments and goodness-of-fit machinery around it.

- 1901 — Aleksandr Lyapunov gives rigorous conditions (the Lyapunov condition) for the Central Limit Theorem, placing the Gaussian's universality on firm analytic ground.

- 1920s–1930s — Lindeberg and Lévy refine the CLT (Lindeberg condition; the i.i.d. Lévy form), giving the modern statements used today.

open-problem

The Gaussian's universality is powerful but bounded, and the boundaries are where modern research lives. Heavy tails: real financial returns, network traffic, and many natural phenomena exhibit tails far heavier than $e^{-x^2/2}$ allows, so Gaussian VaR models systematically underestimate rare crashes; capturing this requires stable, power-law, or extreme-value distributions, and characterising convergence rates (Berry–Esseen bounds) remains delicate when third moments are large. Failure of the CLT: when variances are infinite (e.g. Cauchy-like inputs), sums converge instead to non-Gaussian stable laws — understanding domains of attraction is subtle. High-dimensional geometry: in $n$ dimensions the Gaussian concentrates almost all its mass on a thin shell of radius $\sqrt n$ ("Gaussian annulus"), defying low-dimensional intuition and driving active work in concentration of measure, random matrix theory, and the analysis of high-dimensional algorithms. Dependent data: classical CLTs assume independence; quantifying Gaussian approximation under strong correlations, long memory, or graph-structured dependence is an open and practically vital area.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also