Causal Inference

⚡ Key Takeaways

Causal inference is the science of drawing conclusions about cause-and-effect relationships from data. Its central result: correlation alone cannot establish causation, but with explicit causal assumptions (encoded in graphs or counterfactuals) and proper study design, causal effects can be identified and estimated. Applies wherever we ask "what if?" — medicine, economics, epidemiology, public policy, machine learning. Pioneered by Sewall Wright (1920s), formalized by Donald Rubin, Jerzy Neyman, and Judea Pearl (1970s–2000s).

💡 The Core Insight

Every causal question is a question about counterfactuals: comparing the outcome a unit would have had under treatment with the outcome it would have had under control. Since we can never observe both for the same unit (the fundamental problem of causal inference), causal inference is fundamentally a missing-data problem solved only by combining data with assumptions about how the data were generated. The distinction between $P(Y \mid T=t)$ (seeing) and $P(Y \mid do(T=t))$ (doing) is the entire subject in one symbol.

DefinitionPotential Outcomes (Neyman–Rubin Framework)

For each unit $i$ in a population and a binary treatment $T_i \in \{0,1\}$ , define two potential outcomes:

$Y_i(1)$ — the outcome unit $i$ would exhibit if treated,
$Y_i(0)$ — the outcome unit $i$ would exhibit if untreated.

The individual treatment effect is

1

\quad \tau_i = Y_i(1) - Y_i(0).

Derivation hint: This is a definition — the difference of the two counterfactual outcomes for the same unit. Only one term is ever observed. The observed outcome is linked to potential outcomes by the consistency assumption:

2

\quad Y_i = T_i\, Y_i(1) + (1-T_i)\, Y_i(0).

Derivation hint: Whichever treatment is actually received "switches on" the corresponding potential outcome; the other is counterfactual and unobserved. Symbols: $i$ indexes units; $T_i$ is the realized treatment; $Y_i$ is the realized outcome; $Y_i(t)$ is the potential outcome under treatment level $t$ .

DefinitionAverage Treatment Effect (ATE)

The population-level causal estimand of primary interest is the Average Treatment Effect:

3

\quad \mathrm{ATE} = \mathbb{E}[Y(1) - Y(0)] = \mathbb{E}[Y(1)] - \mathbb{E}[Y(0)].

Derivation hint: Take the expectation of [1] over the population and use linearity of expectation. Related estimands: the Average Treatment effect on the Treated (ATT) $= \mathbb{E}[Y(1)-Y(0)\mid T=1]$ , and the Conditional ATE (CATE) $\tau(x) = \mathbb{E}[Y(1)-Y(0)\mid X=x]$ .

⚠️ The Fundamental Problem of Causal Inference

By [2], for any unit we observe either $Y_i(1)$ or $Y_i(0)$ , never both. Hence $\tau_i$ in [1] is never directly observed for any individual. Causal inference must therefore reconstruct the missing potential outcome using assumptions — it is, structurally, a problem of inferring counterfactuals from factual data.

DefinitionIdentification Assumptions

The ATE is identified (computable from observational data) under three conditions: 1. Ignorability / Unconfoundedness: $\;\{Y(0),Y(1)\} \perp\!\!\!\perp T \mid X$ — given covariates $X$ , treatment is as good as randomly assigned. 2. Positivity / Overlap: $\;0 < P(T=1\mid X=x) < 1$ for all $x$ — every covariate stratum contains both treated and untreated units. 3. SUTVA (Stable Unit Treatment Value Assumption): no interference between units and a single version of each treatment.

TheoremIdentification via Adjustment / Back-door Formula

Under unconfoundedness and positivity, the ATE is identified from observed data by:

4

\quad \mathrm{ATE} = \mathbb{E}_X\!\big[\, \mathbb{E}[Y\mid T=1, X] - \mathbb{E}[Y\mid T=0, X] \,\big].

Derivation hint: Replace $\mathbb{E}[Y(t)\mid X]$ by $\mathbb{E}[Y\mid T=t,X]$ using ignorability + consistency, then average over $X$ . Equivalently, in Pearl's notation, the interventional distribution equals the back-door adjustment formula:

5

\quad P(Y \mid do(T=t)) = \sum_x P(Y \mid T=t, X=x)\, P(X=x).

Derivation hint: Condition on a set $X$ that blocks all back-door paths from $T$ to $Y$ in the causal DAG; intervention severs incoming edges to $T$ , leaving this weighting by the marginal $P(X)$ .

mathematician · Judea Pearl | b. 1936

Israeli-American computer scientist and Turing Award laureate (2011). Pearl developed the do-calculus and the structural-causal-model framework, giving causal inference a graphical algebra. His "ladder of causation" — association, intervention, counterfactuals — organizes the entire field. His book Causality (2000) unified graphical and counterfactual approaches and made $do(\cdot)$ standard notation. Pearl argued that data alone is "profoundly dumb" without a model of how it arose.

🧠 Intuition

Imagine two worlds running in parallel. In World A everyone takes the drug; in World B no one does. The ATE is the difference in average outcome between these two entire worlds. Reality only ever shows us one world, and worse, a mixture: the people who actually took the drug are not a random sample — sicker people may self-select into treatment. This confounding is the villain of the whole subject. Randomization defeats it by making treatment independent of who-you-are; observational adjustment defeats it by conditioning on the variables $X$ that drove the self-selection, effectively comparing like-with-like within strata.

Figure

Confounding: a back-door path from treatment to outcome through a confounder

```svg

```

ExampleEstimating ATE by Stratified Adjustment

Given: A study of a drug ( $T$ ) on recovery ( $Y\in\{0,1\}$ ), with a confounder severity $X\in\{\text{mild, severe}\}$ .

Stratum	$P(X)$	$\mathbb{E}[Y\mid T=1,X]$	$\mathbb{E}[Y\mid T=0,X]$
mild	0.6	0.90	0.80
severe	0.4	0.50	0.30

Steps: 1. Stratum effect (mild): $0.90 - 0.80 = 0.10$ . 2. Stratum effect (severe): $0.50 - 0.30 = 0.20$ . 3. Apply back-door formula [4]: weight by $P(X)$ :

\mathrm{ATE} = 0.6(0.10) + 0.4(0.20) = 0.06 + 0.08 = 0.14.

Boxed answer:

\boxed{\mathrm{ATE} = +0.14 \text{ (a 14 percentage-point increase in recovery probability)}}

Note: a naive unadjusted comparison would mis-weight by the treated/untreated mix and could even reverse sign (Simpson's paradox).

ExampleInverse-Probability Weighting (IPW)

Given: $n=4$ units. Propensity $e(x)=P(T=1\mid X)$ , observed treatment $T$ , outcome $Y$ :

Unit	$e(x)$	$T$	$Y$
1	0.25	1	8
2	0.25	0	4
3	0.80	1	6
4	0.80	0	2

The IPW estimator for $\mathbb{E}[Y(t)]$ :

6

\quad \hat{\mathbb{E}}[Y(1)] = \frac{1}{n}\sum_i \frac{T_i Y_i}{e(X_i)},\qquad \hat{\mathbb{E}}[Y(0)] = \frac{1}{n}\sum_i \frac{(1-T_i) Y_i}{1-e(X_i)}.

Derivation hint: Dividing by the probability of the observed treatment re-creates a pseudo-population in which treatment is independent of $X$ ; expectation of the weighted indicator reproduces the marginal potential-outcome mean.

Steps: 1. $\hat{\mathbb{E}}[Y(1)] = \frac14\!\left(\frac{8}{0.25} + \frac{6}{0.80}\right) = \frac14(32 + 7.5) = 9.875.$ 2. $\hat{\mathbb{E}}[Y(0)] = \frac14\!\left(\frac{4}{0.75} + \frac{2}{0.20}\right) = \frac14(5.333 + 10) = 3.833.$ 3. $\mathrm{ATE} = 9.875 - 3.833 = 6.042.$

Boxed answer:

\boxed{\widehat{\mathrm{ATE}} \approx 6.04 \text{ (outcome units)}}

⚠️ Correlation is not Causation — but neither is "controlling for everything"

Adding more covariates to a regression does not monotonically improve causal estimates. Conditioning on a collider (a common effect of $T$ and $Y$ ) or on a mediator (a variable on the causal path) introduces bias rather than removing it. The correct adjustment set is determined by the causal graph, not by which variables raise $R^2$ or by statistical significance.

CounterexampleCollider / Selection Bias

Suppose talent $T$ and beauty $Y$ are independent in the general population, but both increase the chance of becoming a Hollywood star $C$ (a collider). Among stars (conditioning on $C=1$ ), talent and beauty become negatively correlated — a less talented star must be especially beautiful to have made it. Thus conditioning on a collider manufactures a spurious association between two genuinely independent causes. Blindly "controlling for" $C$ would corrupt an otherwise clean causal estimate.

CounterexampleSimpson's Paradox

A drug improves recovery in both mild and severe patients separately (positive stratum effects), yet appears harmful in the pooled data because severe patients — who recover less often regardless — were disproportionately given the drug. The aggregate association reverses the true causal direction. Only the back-door adjustment [4] (not the raw table) yields the correct conclusion, and which answer is correct depends on whether $X$ is a confounder or a mediator — a causal, not statistical, distinction.

connection

Causal inference sits at the intersection of several fields. It generalizes Regression Analysis, reinterpreting coefficients as causal effects only under explicit assumptions. It relies on Probability Theory and Conditional Expectation for its estimands, and on Directed Acyclic Graphs for encoding qualitative causal knowledge. The d-separation criterion from graph theory determines conditional independencies and valid adjustment sets. Modern estimation borrows from Machine Learning (e.g., causal forests, double/debiased ML) to flexibly model $\mathbb{E}[Y\mid T,X]$ and the propensity score. It is deeply linked to Experimental Design, whose randomization guarantees ignorability by construction.

🔬 Where this appears

⏳ History

- 1921 — Sewall Wright invents path analysis, the first formal method to decompose correlations into causal paths.

- 1923 — Jerzy Neyman, in his master's thesis, introduces potential outcomes for randomized agricultural experiments.

- 1925 — R. A. Fisher formalizes randomization as the basis of experimental inference in The Design of Experiments.

- 1959 — Cornfield et al. publish a sensitivity-analysis argument for smoking causing lung cancer; Bradford Hill (1965) proposes his viewpoints for causation.

- 1974 — Donald Rubin extends potential outcomes to observational studies, founding the Rubin Causal Model.

- 1983 — Rosenbaum & Rubin introduce the propensity score, proving adjustment on it suffices for unconfoundedness.

- 1986 — Robins develops the g-formula and methods for time-varying treatments.

- 1995–2000 — Judea Pearl formalizes do-calculus, structural causal models, and the back-door/front-door criteria; Causality (2000).

- 2018 — Pearl & Mackenzie's The Book of Why popularizes the "ladder of causation"; the era of causal machine learning begins.

open-problem

Unmeasured confounding is the field's central unsolved difficulty: ignorability is untestable from data alone, so all observational estimates rest on a faith assumption. Sensitivity analysis quantifies how strong a hidden confounder would need to be to overturn a conclusion, but cannot prove its absence. Other open frontiers: (i) transportability — when do causal effects estimated in one population transfer to another? (ii) interference / spillovers that violate SUTVA in networks and markets; (iii) reliable CATE estimation with high-dimensional covariates and honest uncertainty; (iv) causal discovery — learning the DAG itself from data, which is provably non-identifiable without further assumptions; and (v) reconciling counterfactual and interventional queries when the structural mechanism is unknown.

Formal Statement

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also