Inference for Means

⚡ Key Takeaways

Inference for means uses sample data to draw conclusions about an unknown population mean μ. The key tool is the t-distribution, which accounts for the extra uncertainty of estimating the standard deviation from the same sample. Applies whenever you want to estimate or test a claim about an average — heights, test scores, drug effects, manufacturing tolerances. Foundational work by W.S. Gosset ("Student") in 1908.

💡 The Core Insight

When σ is known, the standardized sample mean follows a standard normal distribution. When σ is unknown and replaced by s, the denominator becomes random, fattening the tails — the result is the t-distribution. Every confidence interval and test for a mean has the same skeleton: estimate ± (critical value) × (standard error), where the standard error of the mean is $s/\sqrt{n}$ .

DefinitionSampling Distribution of the Sample Mean

Let $X_1, X_2, \dots, X_n$ be a random sample from a population with mean μ and standard deviation σ. The sample mean is

1

\quad \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i

Derivation hint: The sample mean is simply the arithmetic average of the observations — the sum divided by the count. Its expected value and standard deviation (the standard error) are

2

\quad E[\bar{X}] = \mu, \qquad \mathrm{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}}

Derivation hint: Use linearity of expectation for the mean; for the SE, use $\mathrm{Var}(\bar X)=\frac{1}{n^2}\sum\mathrm{Var}(X_i)=\sigma^2/n$ assuming independence, then take the square root.

Symbols: $\bar X$ = sample mean (random variable); $X_i$ = the i-th observation; n = sample size; μ = population mean; σ = population standard deviation; SE = standard error.

TheoremThe t-Statistic

If the population is normally distributed (or n is large), then when σ is unknown and estimated by the sample standard deviation s, the standardized statistic

3

\quad t = \frac{\bar{X} - \mu}{s/\sqrt{n}}

Derivation hint: Replace σ in the z-score by the estimator s; the ratio of a standard normal to an independent scaled chi random variable defines the t-distribution. follows Student's t-distribution with $df = n - 1$ degrees of freedom, where the sample standard deviation is

4

\quad s = \sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(X_i - \bar{X})^2}

Derivation hint: This is the unbiased estimator of σ²; dividing by n−1 (Bessel's correction) compensates for using $\bar X$ in place of the true μ, which "uses up" one degree of freedom.

DefinitionConfidence Interval for a Mean

A level C confidence interval for μ (when σ is unknown) is

5

\quad \bar{x} \pm t^{*}_{n-1}\,\frac{s}{\sqrt{n}}

Derivation hint: Invert the probability statement $P(-t^* < t < t^*) = C$ and solve the inequality for μ, treating $\bar x$ and s as fixed. where $t^{*}_{n-1}$ is the critical value cutting off area $(1-C)/2$ in each tail of the t-distribution with n − 1 degrees of freedom. The quantity $t^{*} \cdot s/\sqrt{n}$ is the margin of error.

DefinitionOne-Sample t-Test

To test the null hypothesis $H_0: \mu = \mu_0$ , compute the test statistic

6

\quad t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}, \qquad df = n-1

Derivation hint: This is the t-statistic of equation [3] evaluated under the assumption that the true mean equals the hypothesized value $\mu_0$ . The p-value is the probability, computed from the $t_{n-1}$ distribution, of obtaining a statistic at least as extreme as the observed t in the direction(s) specified by the alternative hypothesis $H_a$ .

ProofWhy the t-Distribution Replaces the Normal

Step 1. Start from the known-σ case. If the population is normal, then $\bar X \sim N(\mu, \sigma^2/n)$ exactly, so the z-score $Z = (\bar X - \mu)/(\sigma/\sqrt n)$ is standard normal $N(0,1)$ .

Step 2. In practice σ is unknown. We substitute the random estimator s. Now the denominator $s/\sqrt n$ varies from sample to sample — sometimes s underestimates σ, producing unusually large statistics.

Step 3. This extra variability fattens the tails: the statistic $t = (\bar X - \mu)/(s/\sqrt n)$ is more likely to land far from zero than a normal variable would be.

Step 4. Formally, $t$ can be written as $Z / \sqrt{V/(n-1)}$ , where $Z \sim N(0,1)$ and $V = (n-1)s^2/\sigma^2 \sim \chi^2_{n-1}$ are independent. This ratio is precisely the definition of a t-distribution with n − 1 degrees of freedom.

Step 5. As n → ∞, s converges to σ (law of large numbers), the chi-square term concentrates at 1, and the t-distribution converges to the standard normal. Hence for large n the two procedures nearly coincide. ∎

∎

mathematician · William Sealy Gosset ("Student") | 1876–1937

Gosset was a chemist and mathematician employed by the Guinness brewery in Dublin. Working with small samples of barley, hops, and yeast, he needed a way to make reliable inferences when only a handful of measurements were available — exactly the situation where the normal approximation fails. He derived the t-distribution in 1908. Because Guinness forbade employees from publishing (to protect trade secrets), he wrote under the pseudonym "Student", and the distribution has carried that name ever since. R.A. Fisher later supplied the rigorous mathematical foundation and the geometric interpretation in terms of degrees of freedom.

🧠 Intuition

Think of estimating μ as throwing a ring (the confidence interval) at a fixed peg (the true mean). The ring is centered at your sample mean $\bar x$ , and its radius is the margin of error. With a 95% procedure, 95% of all possible rings you might throw would land over the peg — but for any single ring you don't know whether yours is one of them. Why is the t-ring wider than the z-ring? Because you also had to guess the radius from the same throw. When you estimate σ from a small sample, you're uncertain about how wide to make the ring, and the only safe response is to make it a bit wider to compensate. The fewer observations you have, the less you trust s, the more extra width the t-critical value adds — at $df = 1$ , $t^* \approx 12.7$ for 95%, but it shrinks toward $z^* = 1.96$ as n grows.

Figure

Standard normal (z) versus t-distribution: the t-curve has heavier tails

```svg

```

ExampleConfidence Interval for Mean Battery Life

Given: A sample of n = 16 batteries has mean lifetime $\bar x = 42.0$ hours and sample standard deviation $s = 4.8$ hours. Find a 95% confidence interval for the true mean lifetime μ. Assume battery lives are approximately normal.

Step 1 — Degrees of freedom: $df = n - 1 = 15$ .

Step 2 — Critical value: For 95% confidence with $df = 15$ , $t^{*} = 2.131$ (from a t-table).

Step 3 — Standard error:

\mathrm{SE} = \frac{s}{\sqrt{n}} = \frac{4.8}{\sqrt{16}} = \frac{4.8}{4} = 1.20 \text{ hours}

Step 4 — Margin of error:

\text{ME} = t^{*}\cdot \mathrm{SE} = 2.131 \times 1.20 = 2.557 \text{ hours}

Step 5 — Interval:

42.0 \pm 2.557 \;\Rightarrow\; (39.44,\; 44.56)

> Answer: We are 95% confident the true mean battery life is between 39.44 and 44.56 hours.

ExampleOne-Sample t-Test on a Quality Claim

Given: A cereal box is supposed to contain μ₀ = 350 g. A regulator samples n = 25 boxes and finds $\bar x = 347.2$ g with $s = 6.0$ g. At the 5% significance level, is there evidence the mean fill is below 350 g?

Step 1 — Hypotheses: $H_0: \mu = 350$ vs. $H_a: \mu < 350$ (one-sided).

Step 2 — Test statistic:

t = \frac{\bar x - \mu_0}{s/\sqrt n} = \frac{347.2 - 350}{6.0/\sqrt{25}} = \frac{-2.8}{1.20} = -2.333

Step 3 — Degrees of freedom: $df = 24$ .

Step 4 — p-value: For $t = -2.333$ with $df = 24$ (left tail), $p \approx 0.014$ .

Step 5 — Decision: Since $p = 0.014 < 0.05$ , reject $H_0$ . > Answer: There is statistically significant evidence ( $t = -2.33$ , $p \approx 0.014$ ) that the mean fill is less than 350 g. The manufacturer appears to be underfilling.

⚠️ Confidence Is About the Procedure, Not the Interval

A 95% confidence interval does not mean "there is a 95% probability that μ lies in this interval." Once computed, the interval $(39.44, 44.56)$ either contains μ or it does not — there is no probability left. The 95% refers to the long-run success rate of the method: if you repeated the sampling many times, about 95% of the intervals produced would capture μ. Similarly, a p-value is not the probability that $H_0$ is true; it is the probability of data at least this extreme assuming $H_0$ is true.

CounterexampleWhen the t-Procedure Misleads

The t-procedure assumes the sampling distribution of $\bar X$ is approximately normal. Suppose you sample n = 8 incomes from a population with a few extreme billionaires — a strongly right-skewed distribution with outliers. With such a small n, the Central Limit Theorem has not "kicked in," and the single largest value can dominate both $\bar x$ and s. The computed 95% interval may capture the true mean far less than 95% of the time, and a t-test's actual error rate can be much higher than the nominal 5%. The remedy: larger n, transforming the data, or using a method designed for skewed data.

connection

Inference for means sits inside a web of related ideas. The validity of the sample-mean's normality rests on the Central Limit Theorem, which guarantees approximate normality of $\bar X$ for large n regardless of population shape. When σ is known, the procedures use the Normal Distribution and a z-statistic instead of the t. Comparing two group means extends to the Two-Sample t-Test and paired t-tests, while comparing three or more means generalizes to Analysis of Variance. The complementary problem of estimating proportions is handled by Inference for Proportions, which has the same "estimate ± margin of error" skeleton but a different standard error. The general logic of $H_0$ , $H_a$ , and p-values comes from Hypothesis Testing.

🔬 Where this appears

⏳ History

- 1733 — Abraham de Moivre derives the normal curve as an approximation to the binomial, foundation for later sampling theory.

- 1809–1810 — Gauss and Laplace develop the normal distribution and the method of least squares; Laplace proves an early Central Limit Theorem.

- 1877 — Friedrich Helmert derives the chi-square distribution of the sample variance, a key ingredient of the future t-distribution.

- 1908 — W.S. Gosset, writing as "Student," publishes The Probable Error of a Mean in Biometrika, introducing the t-distribution for small samples.

- 1912–1925 — R.A. Fisher provides a rigorous derivation, coins the term "degrees of freedom," and gives the geometric interpretation.

- 1925 — Fisher's Statistical Methods for Research Workers popularizes the t-test and tabulates critical values for practitioners.

- 1933 — Jerzy Neyman and Egon Pearson formalize the theory of hypothesis testing (errors, power), framing the modern decision structure.

- 1937 — Neyman develops the formal theory of confidence intervals, giving the modern interpretation used today.

- Late 20th century — Computers make exact t-computations and simulation-based robustness studies routine, refining guidance on when normality assumptions matter.

open-problem

Several practical questions resist clean answers. How large must n be for the t-procedure to be trustworthy with non-normal data? The classic "n ≥ 30" rule is a crude heuristic — for symmetric distributions even small n works, while for strongly skewed data even n = 50 may be inadequate. Outliers can wreck both $\bar x$ and s; robust alternatives (trimmed means, the bootstrap, rank-based tests) trade some efficiency for resistance. There is ongoing debate over the interpretation and misuse of p-values, culminating in the American Statistical Association's 2016 statement warning against treating $p < 0.05$ as automatic proof. Finally, choosing between frequentist confidence intervals and Bayesian credible intervals remains a genuine philosophical and practical fork: the latter can directly state "there is a 95% probability μ lies in this interval," but only after committing to a prior distribution.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also