Summary Statistics

⚡ Key Takeaways

Summary statistics are single numbers (or small sets of numbers) that capture the essential features of a dataset — its center, spread, and shape. The two big families are measures of central tendency (mean, median, mode) and measures of dispersion (range, variance, standard deviation, IQR). Used everywhere data is collected — science, economics, sports, medicine, polling. Formalised through the 18th–20th centuries by Gauss, Galton, Pearson, and others.

💡 The Core Insight

A dataset has too many numbers to hold in your head. Summary statistics compress that dataset into a few numbers that answer three questions: Where is the data centred? (central tendency), How spread out is it? (dispersion), and What is its shape? (skewness, symmetry). Crucially, this compression is lossy — different datasets can share the same mean and standard deviation while looking completely different. A good analyst always pairs a measure of center with a measure of spread, and ideally with a picture.

DefinitionDataset and Notation

Let a dataset be a list of $n$ numerical observations:

1

\quad x_1, x_2, x_3, \ldots, x_n

Derivation hint: This is just notation — $x_i$ denotes the $i$ -th observation, and $n$ is the total count (the sample size). We distinguish a population (all members of interest, size $N$ ) from a sample (a subset we actually measure, size $n$ ). Population summaries use Greek letters ( $\mu$ , $\sigma$ ); sample summaries use Roman letters ( $\bar{x}$ , $s$ ).

DefinitionMeasures of Central Tendency

The arithmetic mean (average):

2

\quad \bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i = \frac{x_1 + x_2 + \cdots + x_n}{n}

Derivation hint: Add every value, then divide by how many there are; this is the "balance point" of the data. The median $M$ is the middle value when data is sorted in order. For sorted data:

3

\quad M = \begin{cases} x_{(\frac{n+1}{2})} & n \text{ odd} \\[4pt] \dfrac{x_{(\frac{n}{2})} + x_{(\frac{n}{2}+1)}}{2} & n \text{ even} \end{cases}

Derivation hint: Sort the values; the median is the position that splits the lower 50% from the upper 50% — average the two middle ones if there is no single middle. The mode is the value (or values) that appears most frequently. A dataset may be unimodal, bimodal, or have no mode.

DefinitionMeasures of Dispersion

The range is the simplest spread measure:

4

\quad R = x_{\max} - x_{\min}

Derivation hint: Subtract the smallest observation from the largest. The population variance $\sigma^2$ and sample variance $s^2$ :

5

\quad \sigma^2 = \frac{1}{N}\sum_{i=1}^{N}(x_i - \mu)^2, \qquad s^2 = \frac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar{x})^2

Derivation hint: Find each deviation from the mean, square it (so positives and negatives don't cancel), average the squares — but divide the sample version by $n-1$ , not $n$ (Bessel's correction). The standard deviation is the square root of the variance, returning to original units:

6

\quad \sigma = \sqrt{\sigma^2}, \qquad s = \sqrt{s^2}

Derivation hint: Variance is in squared units (e.g. cm²); taking the square root restores the original units (cm) so it can be compared to the data directly. The interquartile range measures the spread of the middle 50%:

7

\quad \text{IQR} = Q_3 - Q_1

Derivation hint: $Q_1$ is the median of the lower half, $Q_3$ the median of the upper half; their difference ignores extreme tails.

TheoremProperties of the Mean and Variance

For any dataset and constants $a, b$ , transforming each value $x_i \to ax_i + b$ gives:

8

\quad \overline{ax+b} = a\bar{x} + b, \qquad s^2_{ax+b} = a^2 s^2, \qquad s_{ax+b} = |a|\,s

Derivation hint: Adding $b$ shifts every value equally so it shifts the mean but not the spread; multiplying by $a$ scales the mean by $a$ and the variance by $a^2$ because variance involves squared deviations.

ProofThe Mean Minimises Squared Deviations

We prove the mean $\bar{x}$ is the value $c$ that minimises the total squared distance $\sum (x_i - c)^2$ .

Step 1. Define the function we wish to minimise:

9

\quad f(c) = \sum_{i=1}^{n}(x_i - c)^2

Derivation hint: Each term measures how far observation $x_i$ is from a candidate center $c$ , squared.

Step 2. Expand and differentiate with respect to $c$ (treating $x_i$ as constants):

10

\quad f'(c) = \sum_{i=1}^{n} 2(x_i - c)(-1) = -2\sum_{i=1}^{n}(x_i - c)

Derivation hint: Apply the chain rule term by term; the derivative of $(x_i - c)^2$ with respect to $c$ is $-2(x_i - c)$ .

Step 3. Set $f'(c) = 0$ to find the minimum:

11

\quad \sum_{i=1}^{n}(x_i - c) = 0 \;\Longrightarrow\; \sum_{i=1}^{n} x_i = nc \;\Longrightarrow\; c = \frac{1}{n}\sum x_i = \bar{x}

Derivation hint: Split the sum, solve for $c$ ; since $f''(c) = 2n > 0$ , this critical point is genuinely a minimum. Hence the mean is the "least-squares center" — no other single number is closer, on average, to all the data in the squared sense. (The median plays the same role for absolute distances $\sum|x_i - c|$ .)

∎

mathematician · Carl Friedrich Gauss | 1777–1855

Gauss developed the method of least squares while predicting the orbit of the asteroid Ceres in 1801, after astronomers lost track of it. His work formalised why the mean is the natural "best estimate" of a true value corrupted by random errors, and led him to the normal (Gaussian) distribution — the bell curve whose two summary statistics, $\mu$ and $\sigma$ , completely determine its shape. His insight that errors cluster symmetrically around a central value underpins the entire modern theory of summary statistics.

🧠 Intuition

Think of the mean as a balance point: if you placed equal weights at each data value along a ruler, the mean is where the ruler balances on a fulcrum. This is why a single billionaire can drag the mean income of a town far above what a typical person earns — that one huge weight is far from the fulcrum and pulls hard. The median, by contrast, is the "middle person in line." It does not care how rich the billionaire is, only that they stand at one end. This is why economists report median household income, not mean — it resists outliers. For spread, imagine darts on a board. Two players might have the same average position (both centred on the bullseye) but one scatters wildly while the other clusters tight. The standard deviation is the typical distance of a dart from the center. A small $s$ means consistency; a large $s$ means unpredictability.

Figure

The mean as balance point, with mean and median diverging on a right-skewed distribution

ExampleTest Scores — Full Summary

Given: Seven students score: 55, 70, 70, 80, 85, 90, 100.

Step 1 — Mean (Eq. 2):

\bar{x} = \frac{55+70+70+80+85+90+100}{7} = \frac{550}{7} \approx 78.6

Step 2 — Median (Eq. 3): Already sorted, $n = 7$ (odd), middle position is $(7+1)/2 = 4$ th value: $M = 80$ .

Step 3 — Mode: The value 70 appears twice; all others once. Mode = 70.

Step 4 — Range (Eq. 4): $R = 100 - 55 = 45$ .

Step 5 — Sample variance (Eq. 5): Deviations from $\bar{x} \approx 78.57$ : squared deviations sum to roughly $(-23.57)^2 + (-8.57)^2 + (-8.57)^2 + (1.43)^2 + (6.43)^2 + (11.43)^2 + (21.43)^2 \approx 1334.3$ .

s^2 = \frac{1334.3}{7-1} \approx 222.4

Step 6 — Standard deviation (Eq. 6): $s = \sqrt{222.4} \approx 14.9$ .

\boxed{\bar{x} \approx 78.6 \text{ pts}, \quad M = 80 \text{ pts}, \quad s \approx 14.9 \text{ pts}}

ExampleThe Outlier Effect on Mean vs Median

Given: Five house prices (in $1000s): 200, 220, 240, 260, 2000 (a mansion).

Step 1 — Mean (Eq. 2):

\bar{x} = \frac{200+220+240+260+2000}{5} = \frac{2920}{5} = 584

Step 2 — Median (Eq. 3): Sorted: 200, 220, 240, 260, 2000. Middle ( $n=5$ ) is the 3rd value: $M = 240$ .

Step 3 — Compare: The mean ( $\$584{,}000$ ) suggests typical homes cost over half a million; the median ( $\$240{,}000$ ) reflects what an ordinary buyer faces. The single mansion inflated the mean by 240% but left the median untouched.

\boxed{\bar{x} = \$584\text{k} \;(\text{misleading}), \qquad M = \$240\text{k} \;(\text{robust})}

⚠️ Common mistake

"The mean is always the best measure of center." False. For skewed data or data with outliers, the mean is dragged toward the tail and misrepresents the typical value. Use the median for income, house prices, and reaction times. "Standard deviation and variance are interchangeable." They convey the same information, but variance is in squared units and cannot be compared to the raw data — always report standard deviation when describing spread in context. Finally, dividing sample variance by $n$ instead of $n-1$ systematically underestimates the true population variance; the $n-1$ (Bessel's correction) compensates for using $\bar{x}$ instead of the unknown $\mu$ .

CounterexampleSame Summaries, Different Data — Anscombe's Quartet

In 1973, statistician Francis Anscombe constructed four datasets that share nearly identical summary statistics: the same mean of $x$ (9.0), same mean of $y$ (7.5), same variance, and same correlation (0.816). Yet when plotted, one is a clean line, one is a curve, one has a single outlier dominating, and one is a vertical cluster with one stray point. This is the definitive proof that summary statistics alone can hide everything important — never trust numbers without also plotting the data. A more recent version, the "Datasaurus Dozen," shows a dinosaur shape and a star sharing identical means and standard deviations.

connection

Summary statistics are the gateway to the rest of statistics. The mean and variance define the Normal Distribution, whose bell shape is fully specified by just $\mu$ and $\sigma$ . Standard deviation underlies the Z-Score, which measures how many standard deviations a value lies from the mean and enables comparison across different scales. The median and quartiles build the Box Plot, a graphical summary of the five-number summary. Variance generalises to Covariance and Correlation, which summarise relationships between two variables rather than one. And the whole framework of summarising samples to infer about populations is the foundation of Statistical Inference and Hypothesis Testing.

🔬 Where this appears

⏳ History

- c. 1654 — Pascal and Fermat lay the groundwork of probability through correspondence on gambling problems, foreshadowing the idea of expected (average) value.

- 1722 — Roger Cotes proposes weighting observations by reliability, an early idea of the weighted mean.

- 1801 — Gauss uses least squares and the normal distribution to recover the orbit of Ceres, cementing the mean as the optimal estimator.

- 1809 — Gauss publishes Theoria Motus, formalising the Gaussian error distribution.

- 1888 — Francis Galton introduces correlation and the concept of regression toward the mean while studying heredity.

- 1894 — Karl Pearson coins the term standard deviation and develops the method of moments, founding the modern statistical vocabulary.

- 1908 — William Gosset ("Student") publishes the t-distribution to handle small samples, addressing why

n-1

matters.

- 1925 — Ronald Fisher's Statistical Methods for Research Workers systematises variance, popularising the analysis of variance (ANOVA).

- 1973 — Francis Anscombe publishes his quartet, warning that summary statistics must always be paired with visualisation.

open-problem

Summary statistics are inherently lossy compression, and choosing which statistics to report is more art than science. For heavily non-normal or multimodal data, no small set of standard summaries faithfully captures the shape — a bimodal distribution (e.g. ages at a wedding: children and adults) has a mean lying in the empty valley between the two peaks. Robust statistics (median, IQR, trimmed means) trade efficiency for resistance to outliers, and there is no universal rule for when to prefer them. Modern debates concern summarising high-dimensional and streaming data, where computing exact medians is expensive and approximate "sketch" algorithms are used. The deepest open question is partly philosophical: how do we summarise data honestly when any single number can be cherry-picked to mislead — a concern at the heart of data ethics and reproducibility.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also