Data Collection

⚡ Key Takeaways

Data collection is the systematic process of gathering, measuring, and recording information about variables of interest. Done well, it produces data that fairly represents a population; done poorly, it introduces bias that no amount of clever analysis can fix. It applies anywhere conclusions are drawn from evidence — science, medicine, government, business, sports. Foundational principles were formalized by statisticians like Ronald Fisher (1920s–30s) and refined through 20th-century polling and survey methodology.

💡 The Core Insight

The goal of data collection is to obtain a sample that faithfully mirrors the population you care about. The single most important tool for achieving this is randomization — using chance to select who or what gets measured. Random selection is the only method that systematically eliminates the silent, invisible biases that creep in whenever humans choose for themselves what to measure.

DefinitionPopulation, Sample, and Census

Population ( $N$ ): the entire group of individuals or items we want to learn about.
Sample ( $n$ ): the subset of the population we actually collect data from.
Census: data collection from every member of the population ( $n = N$ ).
Parameter: a numerical fact about the population (usually unknown), e.g. the true mean $\mu$ .
Statistic: a numerical fact computed from the sample, e.g. the sample mean $\bar{x}$ , used to estimate the parameter.

DefinitionTypes of Sampling Methods

Simple Random Sample (SRS): every possible sample of size $n$ is equally likely.
Stratified Sample: divide the population into homogeneous groups (strata), then take an SRS within each.
Cluster Sample: divide into groups (clusters), randomly select whole clusters, sample everyone within them.
Systematic Sample: choose a random starting point, then select every $k$ -th individual.
Convenience Sample: select whoever is easiest to reach (non-random, biased).
Voluntary Response Sample: subjects choose to participate themselves (non-random, biased).

TheoremThe Representativeness Principle

If a sample is selected by a probability method (every individual has a known, nonzero chance of selection), then as the sample size $n$ increases, sample statistics converge toward the true population parameters. Crucially, no convergence is guaranteed for non-probability samples, regardless of size.

ProofWhy Randomization Defeats Bias

1. Define bias. A method is biased if it systematically over- or under-represents some part of the population, so that the expected value of the statistic differs from the parameter: $E[\bar{x}] \neq \mu$ . 2. Identify the source. Bias enters when selection probability correlates with the variable being measured. Example: phoning landlines under-samples young people, who differ in their answers. 3. Apply randomization. Under an SRS, selection is determined purely by chance — a coin flip or random number — which is independent of any characteristic of the individual. 4. Conclude unbiasedness. Because selection is independent of the measured variable, every subgroup is represented in proportion to its true size on average: $E[\bar{x}] = \mu$ . 5. Note remaining variability. Randomization does not eliminate random sampling error (the natural variation from sample to sample), but this error shrinks predictably as $n$ grows and is captured by a margin of error — unlike bias, which never shrinks. ∎

∎

RA

Ronald A. Fisher

1890–1962

Though a statistician rather than a physicist, Fisher revolutionized data collection through his work at the Rothamsted agricultural research station in England (from 1919). Frustrated by decades of messy farm data, he developed the principles of randomization, replication, and blocking in his 1925 book Statistical Methods for Research Workers and 1935's The Design of Experiments. His insight — that you must design how data is collected before you collect it — transformed experimentation from passive observation into rigorous science. The famous "Lady Tasting Tea" thought experiment, in which a woman claimed to detect whether milk or tea was poured first, became his archetypal lesson in randomizing the order of trials.

🧠 Intuition

Imagine a giant pot of soup. To judge its saltiness, you don't drink the whole pot (a census — wasteful) — you take a spoonful (a sample). But the spoonful only tells you about the whole pot if you stir first. Stirring is randomization: it ensures your spoonful isn't drawn from an unusually salty corner. A bigger spoon (larger $n$ ) gives a slightly more reliable taste, but an unstirred pot will fool you no matter how big the spoon. This is why a tiny, well-stirred sample beats a huge, unstirred one — exactly the lesson of the 1936 election disaster.

Figure

Population, sampling methods, and the bias-vs-error distinction

ExampleSystematic Sampling of a School

Given: A high school has $N = 1200$ students. A researcher wants a sample of $n = 80$ students using systematic sampling.

Steps: 1. Compute the sampling interval using equation [4]:

k = \left\lfloor \frac{1200}{80} \right\rfloor = \lfloor 15 \rfloor = 15

2. Number all students $1$ to $1200$ (e.g., by alphabetical list). 3. Randomly choose a starting point between $1$ and $15$ — say a random draw gives $7$ . 4. Select students numbered $7, 22, 37, 52, \dots$ (add 15 each time). 5. The last selected: $7 + 15(79) = 7 + 1185 = 1192$ ✓ (within range).

Answer: $\boxed{k = 15,\ \text{select students } 7, 22, 37, \ldots, 1192\ (80\text{ students})}$

ExampleMargin of Error for a Poll

Given: A random sample of $n = 1000$ voters finds $p = 0.52$ (52%) support a candidate. Find the 95% margin of error.

Steps: 1. Apply equation [5]:

\text{ME} \approx 1.96\sqrt{\frac{(0.52)(0.48)}{1000}}

2. Compute the variance term: $(0.52)(0.48) = 0.2496$ . 3. Divide by $n$ : $0.2496 / 1000 = 0.0002496$ . 4. Take the square root: $\sqrt{0.0002496} \approx 0.0158$ . 5. Multiply by 1.96: $1.96 \times 0.0158 \approx 0.0310$ .

Answer: $\boxed{\text{ME} \approx 0.031 = 3.1\ \text{percentage points}}$ So the true support is plausibly $52\% \pm 3.1\%$ , i.e., between about $48.9\%$ and $55.1\%$ — note this crosses 50%, so the lead is not statistically decisive.

⚠️ Bigger Is Not Always Better

A common error is believing that a larger sample is automatically more trustworthy. Sample size only controls random error, never bias. A biased collection method (e.g., an online poll where only motivated people respond) produces a systematically wrong answer that gets no closer to the truth as $n$ grows — it just becomes confidently wrong. The 1936 Literary Digest poll's 2.4 million responses lost to Gallup's 50,000 precisely because the Digest's list (car and phone owners) was biased toward wealthier, Republican-leaning voters during the Great Depression.

CounterexampleThe Voluntary Response Trap

A TV news show asks viewers to call a number to vote "yes" or "no" on a controversial law. 100,000 people respond, with 78% voting "no." Is 78% a good estimate of public opinion? No. Only people with strong feelings — especially anger — bother to call (and they can call multiple times). This voluntary response sample systematically over-represents the most motivated, intense opinions. The huge $n$ is worthless; a true SRS of just 1,000 people would give a far more accurate picture. Large size cannot rescue a non-random method.

⚠️ Confusing Observational Studies with Experiments

Collecting data by observing a group (an observational study) can reveal associations but cannot establish causation, because of lurking variables. Only a randomized experiment — where the researcher actively assigns treatments by chance — can establish cause and effect. "Coffee drinkers live longer" (observed) does not mean coffee causes longevity; perhaps coffee drinkers are also wealthier or exercise more.

connection

Data collection is the upstream stage that feeds every other part of statistics. The numbers it produces are summarized by Measures of Central Tendency and Measures of Spread, displayed using Histograms and Frequency Distributions, and interpreted through Probability Distributions. The reliability of any Confidence Interval or Hypothesis Testing result depends entirely on the data having been collected by a valid probability method — these inference tools assume random sampling. The deliberate planning of how to collect data is the subject of Experimental Design, while the chance mechanisms underpinning randomization come from Probability Theory.

🔬 Where this appears

⏳ History

- 1086 CE — The Domesday Book, an early attempted census of England commissioned by William the Conqueror.

- 1790 — The first U.S. Census is conducted, mandated by the Constitution for apportioning representatives.

- 1895 — Norwegian statistician Anders Kiaer proposes the "representative method" of sampling, sparking debate about whether sampling could replace a full census.

- 1906 — W. S. Gosset ("Student"), working at Guinness, develops small-sample methods, recognizing that data collection often involves limited samples.

- 1925–1935 — Ronald Fisher publishes the foundational principles of experimental design: randomization, replication, and blocking.

- 1934 — Jerzy Neyman presents the rigorous theory of stratified random sampling and confidence intervals to the Royal Statistical Society.

- 1936 — George Gallup correctly predicts Roosevelt's victory with a 50,000-person scientific sample, while the Literary Digest's 2.4-million biased poll fails — a landmark demonstration of method over size.

- 1948 — Polls (including Gallup's) wrongly predict Dewey over Truman, exposing flaws in quota sampling and pushing the field toward strict probability sampling.

- 1990s–present — The rise of telephone, then internet and "big data" collection raises new challenges of coverage bias and self-selection in the digital age.

open-problem

Modern data collection faces deep, unresolved challenges. Nonresponse bias is growing: as people screen calls and ignore surveys, response rates have fallen below 10% for many telephone polls, and those who do respond may differ systematically from those who don't — reintroducing bias that randomization was meant to prevent. Big data appears to solve sample-size problems, but massive datasets (social media, web logs) are almost never random samples of any meaningful population, making them prone to systematic error that their size only disguises. The frame problem — obtaining an accurate, complete list of the population to sample from — remains unsolved for hard-to-reach groups (homeless populations, undocumented migrants). And in an era of privacy concerns, techniques like differential privacy deliberately add noise to collected data, forcing statisticians to weigh accuracy against individual confidentiality — an active research frontier.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also