Conditional Probability

⚡ Key Takeaways

Conditional probability measures the likelihood of an event $A$ given that another event $B$ is known to have occurred, written $P(A \mid B)$ . Its defining formula is $P(A \mid B) = P(A \cap B) / P(B)$ . It applies everywhere uncertainty is updated by new information — medical testing, spam filters, gambling, weather forecasts, and machine learning. Formalized by Thomas Bayes (1763) and Pierre-Simon Laplace (1812), it is the backbone of modern probabilistic reasoning.

💡 The Core Insight

When you learn that $B$ occurred, you discard every outcome where $B$ is false and renormalize what remains so the surviving probabilities again sum to 1. Conditional probability is nothing more than restricting attention to the sub-universe of outcomes consistent with your evidence, then asking how much of that sub-universe also satisfies $A$ . The division by $P(B)$ is precisely this renormalization.

DefinitionConditional Probability

Let $A$ and $B$ be events in a sample space $\Omega$ with a probability measure $P$ , and suppose $P(B) > 0$ . The conditional probability of $A$ given $B$ is:

1

\quad P(A \mid B) = \frac{P(A \cap B)}{P(B)}

Derivation hint: Restrict the sample space to $B$ ; the favorable outcomes are those in both $A$ and $B$ , namely $A \cap B$ , and we divide by the "new total" $P(B)$ . Here:

$\Omega$ = the sample space (set of all possible outcomes),
$A, B \subseteq \Omega$ = events,
$A \cap B$ = the event that both $A$ and $B$ occur,
$P(B)$ = the (unconditional) probability of $B$ , required to be strictly positive,
$P(A \mid B)$ = probability of $A$ given that $B$ has occurred.

TheoremBayes' Theorem

For events $A$ and $B$ with $P(A) > 0$ and $P(B) > 0$ :

3

\quad P(A \mid B) = \frac{P(B \mid A)\, P(A)}{P(B)}

Derivation hint: Set the two expressions for $P(A \cap B)$ in equation [2] equal and solve for $P(A \mid B)$ . When the denominator must be expanded over a partition $\{A_1, A_2, \dots, A_n\}$ of $\Omega$ (the law of total probability):

4

\quad P(B) = \sum_{i=1}^{n} P(B \mid A_i)\, P(A_i)

Derivation hint: The events $B \cap A_i$ are disjoint and their union is $B$ ; apply the multiplication rule [2] to each term $P(B \cap A_i)$ .

TheoremIndependence

Events $A$ and $B$ are independent if and only if:

5

\quad P(A \cap B) = P(A)\, P(B) \quad\Longleftrightarrow\quad P(A \mid B) = P(A)

Derivation hint: Substitute $P(A \cap B) = P(A)P(B)$ into [1]; the $P(B)$ cancels, leaving $P(A \mid B) = P(A)$ — knowing $B$ tells you nothing about $A$ .

ProofJustifying the Definition via Renormalization

We show that equation [1] is the unique sensible definition for "probability of $A$ within the world where $B$ holds." 1. Restrict the universe. Given that $B$ occurred, only outcomes inside $B$ remain possible. We seek a new probability measure $P_B$ defined on subsets of $B$ . 2. Proportionality. Within $B$ , relative likelihoods should be preserved — if outcome $x$ was twice as likely as $y$ before, it still is. So $P_B(A) = c \cdot P(A \cap B)$ for some constant $c$ , since only the part of $A$ inside $B$ matters. 3. Normalization. A valid probability requires $P_B(B) = 1$ . Thus $1 = c \cdot P(B \cap B) = c \cdot P(B)$ , giving $c = 1/P(B)$ . 4. Conclude. Therefore $P_B(A) = \dfrac{P(A \cap B)}{P(B)}$ , which is exactly $P(A \mid B)$ in equation [1]. $\blacksquare$

∎

ProofThat Conditional Probability Is Itself a Valid Probability

Fix $B$ with $P(B) > 0$ . We verify $P(\cdot \mid B)$ satisfies the Kolmogorov axioms. 1. Non-negativity: $P(A \mid B) = P(A\cap B)/P(B) \ge 0$ since both factors are non-negative. 2. Total mass: $P(\Omega \mid B) = P(\Omega \cap B)/P(B) = P(B)/P(B) = 1$ . 3. Countable additivity: For disjoint $A_1, A_2, \dots$ , the sets $A_i \cap B$ are also disjoint, so

P\!\left(\bigcup_i A_i \,\middle|\, B\right) = \frac{P\!\left(\bigcup_i (A_i \cap B)\right)}{P(B)} = \frac{\sum_i P(A_i \cap B)}{P(B)} = \sum_i P(A_i \mid B).

Hence $P(\cdot \mid B)$ is a genuine probability measure. $\blacksquare$

∎

mathematician · Thomas Bayes | c. 1701–1761

An English Presbyterian minister and amateur mathematician, Bayes left an unpublished essay solving the "inverse probability" problem: given observed outcomes, how do we infer the probability of an underlying cause? His friend Richard Price edited and published it posthumously in 1763 as An Essay towards solving a Problem in the Doctrine of Chances. Bayes considered a thought experiment of balls rolled on a table — reasoning backward from where later balls land to where an unseen first ball lies. Though he never wrote the theorem in its modern form, the principle of reversing conditionals — turning $P(B \mid A)$ into $P(A \mid B)$ — bears his name and underlies an entire school of statistical inference.

🧠 Intuition

Picture the sample space as a rectangle of total area 1. Event $B$ is a region inside it; event $A$ is another region, possibly overlapping. The unconditional $P(A)$ is the fraction of the whole rectangle covered by $A$ . But $P(A \mid B)$ throws away everything outside $B$ and asks: of the area now remaining (just $B$ ), what fraction is also $A$ ? That fraction is the overlap $A \cap B$ divided by all of $B$ . This is why conditioning can raise or lower a probability. If $A$ overlaps $B$ heavily, learning $B$ makes $A$ more likely. If they barely overlap, $B$ makes $A$ less likely. And if $A$ 's share of $B$ equals its share of the whole space, then $B$ is irrelevant — that is independence.

Figure

Conditional probability as renormalization: restricting the universe to B

ExampleTwo Dice — The Sum Given a Six

Given: Roll two fair six-sided dice. Let $A$ = "the sum is 8" and $B$ = "at least one die shows a 6." Find $P(A \mid B)$ .

Step 1 — Count $B$ . Outcomes with at least one 6: there are 11 (the six $(6, x)$ plus the six $(x, 6)$ , minus the double-counted $(6,6)$ ). So $P(B) = 11/36$ .

Step 2 — Count $A \cap B$ . Sum 8 and a six: only $(6,2)$ and $(2,6)$ . So $P(A \cap B) = 2/36$ .

Step 3 — Apply equation [1].

P(A \mid B) = \frac{2/36}{11/36} = \frac{2}{11}.

Answer:

\boxed{P(A \mid B) = \tfrac{2}{11} \approx 0.182}

Note that unconditionally $P(A) = 5/36 \approx 0.139$ , so learning "at least one six" raised the probability of an 8.

ExampleMedical Test — Bayes in Action

Given: A disease affects 1% of a population: $P(D) = 0.01$ . A test has sensitivity $P(+ \mid D) = 0.99$ and false-positive rate $P(+ \mid \neg D) = 0.05$ . A person tests positive. Find $P(D \mid +)$ .

Step 1 — Total probability of a positive (equation [4]).

P(+) = P(+\mid D)P(D) + P(+\mid \neg D)P(\neg D) = (0.99)(0.01) + (0.05)(0.99).

P(+) = 0.0099 + 0.0495 = 0.0594.

Step 2 — Apply Bayes' theorem (equation [3]).

P(D \mid +) = \frac{P(+\mid D)\,P(D)}{P(+)} = \frac{0.0099}{0.0594}.

Answer:

\boxed{P(D \mid +) \approx 0.167 = 16.7\%}

Despite a 99% sensitive test, a positive result means only a ~17% chance of disease, because the disease is rare. This is the famous base-rate effect.

⚠️ Do Not Confuse

P(A \mid B)

with

P(B \mid A)

These are generally not equal. In the medical example, $P(+ \mid D) = 0.99$ but $P(D \mid +) \approx 0.17$ . Swapping them is the prosecutor's fallacy: "the chance of this DNA match if innocent is 1 in a million, therefore the chance the defendant is innocent is 1 in a million" is invalid reasoning that ignores the base rate of innocent people. Equation [3] is precisely the correct way to convert one into the other.

⚠️ Conditioning on a Zero-Probability Event

Equation [1] requires $P(B) > 0$ . If $P(B) = 0$ , the ratio $0/0$ is undefined and $P(A \mid B)$ has no meaning from the elementary definition. Handling such cases (e.g., conditioning on a continuous variable taking an exact value) requires the more advanced machinery of conditional densities and measure theory.

CounterexampleIndependence Is Not "Mutual Exclusivity"

A common error is thinking disjoint (mutually exclusive) events are independent. They are nearly the opposite. If $A$ and $B$ are disjoint with positive probabilities, then $A \cap B = \varnothing$ , so $P(A \mid B) = 0 \neq P(A)$ . Knowing $B$ occurred tells you $A$ cannot have — that is maximal dependence, not independence. For example, "rolling a 2" and "rolling a 5" on one die are mutually exclusive but strongly dependent.

connection

Conditional probability is the hinge connecting many topics. It generalizes Joint Probability via the multiplication rule [2], and it specializes to ordinary probability when the condition is $\Omega$ . It defines Statistical Independence (equation [5]) and powers Bayes' Theorem for inference. The Law of Total Probability (equation [4]) decomposes events over a partition, and conditional probability underlies Conditional Expectation, Markov Chains (where the future depends only on the present state), and Random Variables through conditional distributions. It is also the foundation of Decision Theory under uncertainty.

🔬 Where this appears

⏳ History

- 1654 — Blaise Pascal and Pierre de Fermat correspond about the "problem of points," laying groundwork for probability and implicitly conditional reasoning.

- 1713 — Jacob Bernoulli's Ars Conjectandi establishes the law of large numbers, framing probability as quantified uncertainty.

- 1763 — Thomas Bayes' essay is published posthumously by Richard Price, introducing inverse probability.

- 1774 & 1812 — Pierre-Simon Laplace independently develops and greatly extends Bayesian ideas in Théorie analytique des probabilités, giving the rule its general form.

- 1837 — Siméon Poisson clarifies conditional reasoning in his work on judgments and probability.

- 1933 — Andrey Kolmogorov's Grundbegriffe der Wahrscheinlichkeitsrechnung axiomatizes probability, defining conditional probability rigorously and extending it to general (measure-theoretic) conditioning.

- 1950s — Conditional probability becomes central to information theory (Shannon) and statistical decision theory (Wald).

- 1980s–1990s — Judea Pearl develops Bayesian networks, making conditional dependence a computational engine for AI.

- 2000s–present — Bayesian methods dominate machine learning, genomics, and data science.

open-problem

Conditioning on events of probability zero — for example, "given that a randomly chosen point lies exactly on a line" — cannot be handled by equation [1] and leads to the Borel–Kolmogorov paradox, where the answer depends on how one approaches the limiting condition. The measure-theoretic resolution defines conditional expectation as a Radon–Nikodym derivative, but it is only specified "almost everywhere," leaving genuine philosophical ambiguity about conditioning on individual zero-probability outcomes. Foundational debates between frequentist and Bayesian interpretations of what $P(A \mid B)$ means — a long-run frequency versus a degree of belief — remain unresolved and actively shape statistical practice today.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also