Independence

⚡ Key Takeaways

Independence is the property that the occurrence of one event gives no information about another. Two events $A$ and $B$ are independent precisely when $P(A \cap B) = P(A)\,P(B)$ . It applies across probability theory, statistics, and any field modelling randomness — from coin tosses to genetics. The concept was formalized by mathematicians such as Abraham de Moivre (1718) and Andrei Kolmogorov (1933).

💡 The Core Insight

Independence means multiplication works. When events are independent, the probability of a combination is the product of the parts — no correction factor, no information leaking between them. Equivalently, conditioning on one event leaves the other's probability untouched: $P(A\mid B) = P(A)$ . All of the subject's power flows from this single multiplicative rule.

DefinitionIndependence of Two Events

Let $(\Omega, \mathcal{F}, P)$ be a probability space with events $A, B \in \mathcal{F}$ . The events $A$ and $B$ are independent if and only if

1

\quad P(A \cap B) = P(A)\,P(B)

Derivation hint: This is a definition, not a derived result; it is the formal encoding of "joint probability factors into the product of marginals." Here:

$\Omega$ = the sample space (set of all outcomes),
$\mathcal{F}$ = the collection of events (subsets of $\Omega$ ),
$P$ = the probability measure, $P:\mathcal{F}\to[0,1]$ ,
$A \cap B$ = the event "both $A$ and $B$ occur,"
$P(A)$ , $P(B)$ = the marginal probabilities of each event.

DefinitionConditional Form

If $P(B) > 0$ , independence is equivalent to

2

\quad P(A \mid B) = P(A)

Derivation hint: Start from the definition of conditional probability $P(A\mid B)=P(A\cap B)/P(B)$ and substitute equation [1]. where $P(A\mid B)$ is the conditional probability of $A$ given that $B$ has occurred. This says: learning $B$ happened does not revise the probability of $A$ .

DefinitionMutual Independence of Several Events

Events $A_1, A_2, \ldots, A_n$ are mutually independent if for every subset $\{i_1, \ldots, i_k\} \subseteq \{1, \ldots, n\}$ ,

3

\quad P\!\left(A_{i_1} \cap \cdots \cap A_{i_k}\right) = \prod_{j=1}^{k} P\!\left(A_{i_j}\right)

Derivation hint: Mutual independence requires the product rule to hold for all subcollections — pairwise independence (only $k=2$ ) is strictly weaker and does not imply this.

DefinitionIndependence of Random Variables

Random variables $X$ and $Y$ are independent if for all real numbers $x, y$ ,

4

\quad P(X \le x,\ Y \le y) = P(X \le x)\,P(Y \le y)

Derivation hint: Apply equation [1] to the events $\{X\le x\}$ and $\{Y\le y\}$ ; equivalently the joint cumulative distribution function factors into the product of the marginals. For discrete variables this is equivalent to the joint probability mass function factoring, and for continuous variables to the joint density factoring:

5

\quad f_{X,Y}(x,y) = f_X(x)\,f_Y(y)

Derivation hint: Differentiate the factored joint CDF in equation [4] with respect to both $x$ and $y$ .

TheoremConsequences of Independence

If $X$ and $Y$ are independent random variables, then:

6

\quad E[XY] = E[X]\,E[Y]

Derivation hint: Expand $E[XY]=\iint xy\,f_{X,Y}(x,y)\,dx\,dy$ , substitute the factored density from equation [5], and separate into a product of single integrals.

7

\quad \mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)

Derivation hint: Use $\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)+2\,\mathrm{Cov}(X,Y)$ and note that independence forces $\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]=0$ via equation [6].

ProofConditional Form Equivalence

We show equation [1] $\Leftrightarrow$ equation [2] when $P(B)>0$ . 1. By the definition of conditional probability, $P(A\mid B) = \dfrac{P(A\cap B)}{P(B)}$ . 2. Assume independence, equation [1]: $P(A\cap B) = P(A)P(B)$ . 3. Substitute into step 1: $P(A\mid B) = \dfrac{P(A)P(B)}{P(B)} = P(A)$ , giving equation [2]. 4. Conversely, assume $P(A\mid B)=P(A)$ . Multiply both sides of step 1's definition by $P(B)$ : $P(A\cap B) = P(A\mid B)\,P(B) = P(A)\,P(B)$ , recovering equation [1]. 5. Therefore the two characterizations are equivalent. $\blacksquare$

∎

ProofMultiplicativity of Expectation

We derive equation [6] for independent continuous variables. 1. By definition, $E[XY] = \displaystyle\iint_{\mathbb{R}^2} xy\,f_{X,Y}(x,y)\,dx\,dy$ . 2. By independence, equation [5] gives $f_{X,Y}(x,y) = f_X(x)f_Y(y)$ . 3. Substitute: $E[XY] = \displaystyle\iint xy\,f_X(x)f_Y(y)\,dx\,dy$ . 4. The integrand separates: $= \left(\displaystyle\int x\,f_X(x)\,dx\right)\left(\displaystyle\int y\,f_Y(y)\,dy\right)$ . 5. Each factor is a marginal expectation: $= E[X]\,E[Y]$ . $\blacksquare$

∎

mathematician · Andrei Kolmogorov | 1903–1987

The Soviet mathematician Andrei Kolmogorov gave probability theory its rigorous axiomatic foundation in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung. Within that framework, independence is not an afterthought but a defining structural concept — Kolmogorov elevated it to a precise measure-theoretic notion via the factorization of probabilities. His treatment unified centuries of informal "multiplying chances" into the clean definition used today, and his zero-one law shows how independence governs the behaviour of infinite sequences of random events.

🧠 Intuition

Imagine two separate coin flips. Knowing the first landed heads tells you absolutely nothing about the second — that's independence. The "multiply the probabilities" rule works because each outcome of the first coin gets paired with each outcome of the second, with no preference. Think of a grid: the rows are outcomes of $A$ , the columns are outcomes of $B$ . Independence means the probability of any cell is just (row probability) × (column probability) — the grid is "rectangular," with no clumping. Dependence warps the grid: some cells get extra weight, some get less, and learning the column shifts your belief about the row. A useful mental check: if I tell you $B$ happened, does your bet on $A$ change? If not, they're independent.

WP

Why physicists care

In statistical mechanics, independence of particle states underlies the factorization of the partition function and the multiplicative structure of probabilities for non-interacting systems. When particles don't interact, their joint configuration probability factorizes exactly as in equation [3] — and the breakdown of independence (correlations) is precisely what signals interactions, phase transitions, and entanglement in quantum systems.

ExampleTwo Dice

Given: Roll two fair six-sided dice. Let $A$ = "first die shows 4," $B$ = "second die shows an even number." Are $A$ and $B$ independent? Find $P(A\cap B)$ .

Steps: 1. $P(A) = \dfrac{1}{6}$ (one face out of six). 2. $P(B) = \dfrac{3}{6} = \dfrac{1}{2}$ (faces 2, 4, 6). 3. The dice are physically separate, so we test equation [1]. Count outcomes: $A\cap B$ requires first die = 4 (1 way) and second die even (3 ways) = 3 outcomes out of 36. 4. $P(A\cap B) = \dfrac{3}{36} = \dfrac{1}{12}$ . 5. Check: $P(A)P(B) = \dfrac{1}{6}\cdot\dfrac{1}{2} = \dfrac{1}{12}$ . ✓ Equal, so independent.

\boxed{P(A\cap B) = \tfrac{1}{12},\quad \text{events are independent (dimensionless)}}

ExampleDrawing Cards Without Replacement

Given: From a standard 52-card deck, draw two cards without replacement. Let $A$ = "first card is an Ace," $B$ = "second card is an Ace." Are they independent?

Steps: 1. $P(A) = \dfrac{4}{52} = \dfrac{1}{13}$ . 2. $P(B)$ overall (by symmetry / law of total probability) $= \dfrac{4}{52} = \dfrac{1}{13}$ . 3. Compute $P(A\cap B)$ : $P(A)\cdot P(B\mid A) = \dfrac{4}{52}\cdot\dfrac{3}{51} = \dfrac{12}{2652} = \dfrac{1}{221}$ . 4. Test equation [1]: $P(A)P(B) = \dfrac{1}{13}\cdot\dfrac{1}{13} = \dfrac{1}{169}$ . 5. Since $\dfrac{1}{221} \neq \dfrac{1}{169}$ , the events are dependent. Drawing one Ace lowers the chance of a second.

\boxed{P(A\cap B) = \tfrac{1}{221} \neq \tfrac{1}{169} = P(A)P(B)\ \Rightarrow\ \text{dependent}}

Figure

Probability grid: independent vs. dependent events

⚠️ Independent ≠ Mutually Exclusive

A frequent error is confusing independence with mutual exclusivity (disjointness). Mutually exclusive events $A$ and $B$ satisfy $P(A\cap B)=0$ — they cannot both happen. But if both have positive probability, this forces dependence: knowing $A$ occurred guarantees $B$ did not, so $P(B\mid A)=0\neq P(B)$ . Mutually exclusive events with nonzero probability are about as dependent as events can be.

⚠️ Pairwise ≠ Mutual Independence

Equation [3] demands the product rule for all subsets. Three events can be pairwise independent (every pair satisfies equation [1]) yet fail to be mutually independent, because the triple product rule breaks. Always check the full hierarchy of subsets when more than two events are involved.

CounterexamplePairwise but not Mutually Independent

Toss two fair coins. Define:

$A$ = "first coin is heads," $P(A)=\tfrac12$
$B$ = "second coin is heads," $P(B)=\tfrac12$
$C$ = "the two coins match (both H or both T)," $P(C)=\tfrac12$

Each pair is independent: e.g. $P(A\cap C) = P(\text{HH}) = \tfrac14 = P(A)P(C)$ , and similarly for the others. But $A\cap B\cap C = \{\text{HH}\}$ , so $P(A\cap B\cap C)=\tfrac14$ , whereas $P(A)P(B)P(C)=\tfrac18$ . Since $\tfrac14\neq\tfrac18$ , the three are not mutually independent — indeed, knowing $A$ and $B$ completely determines $C$ .

⚠️ Uncorrelated ≠ Independent

Independence implies zero correlation (equation [6] forces $\mathrm{Cov}=0$ ), but the converse fails. Two variables can have zero covariance yet be strongly dependent — for example, $X$ uniform on $[-1,1]$ and $Y=X^2$ have $\mathrm{Cov}(X,Y)=0$ but $Y$ is completely determined by $X$ . Correlation only measures linear association.

connection

Independence is the keystone linking many areas. It is defined through Conditional Probability (equation [2]) and gives the clean form of Bayes' Theorem when priors and likelihoods factor. It powers the Binomial Distribution, which counts successes in independent trials, and is the crucial hypothesis behind the Central Limit Theorem and the Law of Large Numbers. In statistics it underlies the assumption of Random Sampling. The failure of independence is quantified by Covariance and Correlation, and its multivariate structure appears in the Joint Probability Distribution.

🔬 Where this appears

⏳ History

- 1654 — Pascal and Fermat correspond on games of chance, implicitly multiplying probabilities of independent rolls.

- 1657 — Christiaan Huygens publishes De ratiociniis in ludo aleae, the first printed probability treatise, using product reasoning.

- 1713 — Jacob Bernoulli's Ars Conjectandi (posthumous) develops the Law of Large Numbers for independent Bernoulli trials.

- 1718 — Abraham de Moivre's The Doctrine of Chances states the multiplication rule for independent events explicitly.

- 1733 — De Moivre derives the normal approximation to the binomial, an early Central Limit result for independent trials.

- 1812 — Pierre-Simon Laplace's Théorie analytique des probabilités systematizes conditional and independent probability.

- 1900 — Karl Pearson develops correlation, sharpening the distinction between dependence and independence.

- 1933 — Andrei Kolmogorov axiomatizes probability, giving independence its modern measure-theoretic definition (equation [1]) and proving the zero-one law.

- 1950s onward — Independence assumptions become central to mathematical statistics, information theory (Shannon), and stochastic processes.

open-problem

Independence is mathematically crisp but subtle in application. Testing for independence from finite data is genuinely hard: standard tests (chi-square, distance correlation) can fail to detect complex nonlinear dependence, and "independence" is never proven, only failed-to-reject. In high-dimensional settings, distinguishing true independence from spurious correlations is an active research area. Deeper still, the notion of conditional independence — central to graphical models, causal inference, and Bayesian networks — generates rich combinatorial structure, and characterizing which conditional independence relations are simultaneously realizable remains partly open. Finally, in foundations, the relationship between statistical independence and causal independence is philosophically and practically unresolved: independence is symmetric and observational, while causation is directional, so correlation/dependence can never by itself establish or refute a causal link.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also