Probability Models

⚡ Key Takeaways

A probability model is a precise mathematical description of a random experiment, consisting of a sample space of all possible outcomes together with a rule assigning probabilities to events. Key result: every valid model obeys the three Kolmogorov axioms — probabilities are non-negative, the whole sample space has probability 1, and the probability of mutually exclusive events adds. Used everywhere uncertainty appears: dice and cards, genetics, insurance, quality control, polling, and physics. Formalized by Andrey Kolmogorov in 1933, building on centuries of gambling mathematics.

💡 The Core Insight

Randomness becomes mathematics the moment you commit to two things: what can happen (the sample space) and how the total probability of 1 is distributed across what can happen. Everything else — odds, expected values, conditional probabilities — is bookkeeping built on this foundation. A probability model is therefore not a guess about the future; it is a complete accounting system for uncertainty in which all the probabilities are forced to balance to 1.

DefinitionSample Space and Events

The sample space $S$ is the set of all possible outcomes of a random experiment. An individual outcome is a sample point. An event $A$ is any subset of $S$ ; it "occurs" if the actual outcome belongs to $A$ .

$S$ — the entire sample space (the certain event)
$\varnothing$ — the empty set (the impossible event)
$A, B \subseteq S$ — events
$A \cup B$ — " $A$ or $B$ " (union)
$A \cap B$ — " $A$ and $B$ " (intersection)
$A^c = S \setminus A$ — "not $A$ " (complement)

Two events are mutually exclusive (disjoint) if $A \cap B = \varnothing$ — they cannot both occur.

DefinitionProbability Function and the Kolmogorov Axioms

A probability function $P$ assigns to each event $A$ a real number $P(A)$ satisfying three axioms:

1

\quad P(A) \ge 0 \quad \text{for every event } A

Derivation hint: This is an axiom (a chosen rule); it encodes that probability is a measure of "amount of certainty," which cannot be negative.

2

\quad P(S) = 1

Derivation hint: An axiom expressing that some outcome in $S$ is certain to occur; it normalizes the total probability.

3

\quad P(A_1 \cup A_2 \cup \cdots) = \sum_{i} P(A_i) \quad \text{when the } A_i \text{ are pairwise mutually exclusive}

Derivation hint: The countable additivity axiom; disjoint events contribute independently to the total, so their probabilities simply sum.

TheoremConsequences of the Axioms

From the three axioms alone, the following must hold for all events $A, B$ :

4

\quad P(A^c) = 1 - P(A)

Derivation hint: $A$ and $A^c$ are disjoint with union $S$ ; apply [3] then [2].

5

\quad P(\varnothing) = 0

Derivation hint: Set $A = S$ in [4], since $S^c = \varnothing$ .

6

\quad 0 \le P(A) \le 1

Derivation hint: Lower bound from [1]; upper bound because $P(A) = 1 - P(A^c) \le 1$ as $P(A^c)\ge 0$ .

7

\quad P(A \cup B) = P(A) + P(B) - P(A \cap B)

Derivation hint: The inclusion–exclusion principle; adding $P(A)$ and $P(B)$ double-counts the overlap $A\cap B$ , so subtract it once.

8

\quad \text{If } A \subseteq B \text{ then } P(A) \le P(B)

Derivation hint: Write $B = A \cup (B \setminus A)$ as a disjoint union and apply [3] with [1].

DefinitionEqually Likely Outcomes (Classical Model)

When $S$ is finite with $n$ outcomes that are all equally likely, the probability of any event $A$ containing $k$ favorable outcomes is

9

\quad P(A) = \frac{|A|}{|S|} = \frac{\text{number of favorable outcomes}}{\text{total number of outcomes}} = \frac{k}{n}

Derivation hint: By symmetry each outcome has probability $1/n$ ; by [3] the event's probability is the sum of $k$ such terms.

ProofDeriving the Complement and Inclusion–Exclusion Rules

We derive equations [4] and [7] from the axioms.

Step 1. For any event $A$ , the events $A$ and $A^c$ are mutually exclusive (they share no outcome) and their union is all of $S$ .

Step 2. By additivity [3]: $P(A \cup A^c) = P(A) + P(A^c)$ .

Step 3. Since $A \cup A^c = S$ and $P(S) = 1$ by [2], we get $P(A) + P(A^c) = 1$ , hence $P(A^c) = 1 - P(A)$ . This is [4].

Step 4. Now for [7], decompose $A \cup B$ into three disjoint pieces: "only $A$ " = $A \cap B^c$ , "only $B$ " = $A^c \cap B$ , and "both" = $A \cap B$ .

Step 5. By additivity, $P(A \cup B) = P(A\cap B^c) + P(A^c \cap B) + P(A \cap B)$ .

Step 6. Also $P(A) = P(A\cap B^c) + P(A\cap B)$ and $P(B) = P(A^c \cap B) + P(A \cap B)$ , again by decomposing each into disjoint parts.

Step 7. Add the two equations in Step 6: $P(A) + P(B) = P(A\cap B^c) + P(A^c\cap B) + 2P(A\cap B)$ .

Step 8. Subtract $P(A\cap B)$ from both sides and compare with Step 5: $P(A)+P(B)-P(A\cap B) = P(A\cup B)$ . This is [7]. $\blacksquare$

∎

mathematician · Andrey Kolmogorov | 1903–1987

The Soviet mathematician Andrey Nikolaevich Kolmogorov gave probability its modern rigorous foundation in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). Before him, probability rested on intuitive notions of "equally likely cases" and frequency, which broke down for continuous and infinite situations. Kolmogorov's insight was to identify probability with measure theory: a sample space, a collection of measurable events (a σ-algebra), and a measure of total mass 1. This abstraction unified discrete and continuous probability and let mathematicians prove deep results rigorously. He also made fundamental contributions to turbulence, dynamical systems, and information theory.

🧠 Intuition

Picture a probability model as one liter of paint to be spread over a map of all possible outcomes. The map is the sample space; you must use exactly one liter (axiom [2]); you cannot apply negative paint (axiom [1]); and the paint over a region equals the sum of paint over its non-overlapping subregions (axiom [3]). An "event" is just an area on the map, and its probability is how much paint covers it. The complement rule [4] says the paint off a region equals one liter minus the paint on it. Inclusion–exclusion [7] is the obvious statement that when two regions overlap, adding their paint double-counts the overlap. Choosing a model means deciding how thickly to spread the paint — uniformly (classical model), or in some uneven pattern that better matches reality.

Figure

A probability model: total mass 1 distributed over the sample space, with two overlapping events

ExampleDrawing a Card

Given: A standard 52-card deck is well shuffled; one card is drawn. Let $A$ = "the card is a heart" and $B$ = "the card is a face card (J, Q, K)". Find $P(A \cup B)$ .

Step 1 — Model. Sample space $S$ = the 52 cards, all equally likely, so use [9] with $n = 52$ .

Step 2 — Count favorables. Hearts: $|A| = 13$ , so $P(A) = 13/52$ . Face cards: $|B| = 12$ (3 per suit $\times$ 4 suits), so $P(B) = 12/52$ .

Step 3 — Overlap. Hearts that are also face cards: J♥, Q♥, K♥, so $|A \cap B| = 3$ and $P(A\cap B) = 3/52$ .

Step 4 — Inclusion–exclusion [7].

P(A \cup B) = \tfrac{13}{52} + \tfrac{12}{52} - \tfrac{3}{52} = \tfrac{22}{52} = \tfrac{11}{26}.

\boxed{P(A \cup B) = \tfrac{11}{26} \approx 0.423}

ExampleA Biased (Non-Uniform) Model

Given: A spinner has 3 outcomes — Red, Green, Blue — with $P(\text{Red}) = 0.5$ , $P(\text{Green}) = 0.3$ . Outcomes are mutually exclusive and exhaustive. Find $P(\text{Blue})$ and $P(\text{not Red})$ .

Step 1 — Use normalization [2]. Probabilities must sum to 1:

P(\text{Red}) + P(\text{Green}) + P(\text{Blue}) = 1.

Step 2 — Solve for Blue.

P(\text{Blue}) = 1 - 0.5 - 0.3 = 0.2.

Step 3 — Complement rule [4].

P(\text{not Red}) = 1 - P(\text{Red}) = 1 - 0.5 = 0.5.

This is not a classical equally-likely model — the outcomes have unequal weights, yet the axioms still hold and the total is exactly 1.

\boxed{P(\text{Blue}) = 0.2, \qquad P(\text{not Red}) = 0.5}

⚠️ Common mistake

Probabilities are not always $1/n$ . Equation [9] applies only when outcomes are equally likely. A common error is to list the sample space and assume uniform probabilities without justification. For the sum of two dice, the outcomes $\{2, 3, \dots, 12\}$ are not equally likely — a 7 is six times as likely as a 2 — so the correct model uses the 36 equally likely ordered pairs, not the 11 sums. Always check whether your chosen outcomes deserve equal weight.

⚠️ Common mistake

The "gambler's fallacy." A probability model for independent trials carries no memory. After five heads in a row, a fair coin still has $P(\text{heads}) = 1/2$ on the next flip. The coin does not "owe" you tails. Past outcomes do not change the model's probabilities for future independent trials.

CounterexampleAdditivity Fails Without Disjointness

Suppose $P(\text{rain Saturday}) = 0.6$ and $P(\text{rain Sunday}) = 0.6$ . It is tempting to write $P(\text{rain at least one day}) = 0.6 + 0.6 = 1.2$ — but probabilities can never exceed 1 (by [6]). The error: axiom [3] requires mutually exclusive events, and "rain Saturday" and "rain Sunday" can both happen. The correct tool is [7], which subtracts the overlap $P(\text{rain both days})$ , keeping the answer $\le 1$ .

connection

A probability model is the launching point for nearly all of probability and statistics. Once the model assigns probabilities, Conditional Probability asks how probabilities update given partial information, and Independence of Events characterizes when one event carries no information about another. Attaching numerical values to outcomes produces a Random Variable, whose long-run average is the Expected Value and whose spread is the Variance. Specific models — like the Binomial Distribution for repeated independent trials or the Normal Distribution for continuous measurements — are particular probability models with named structure. The classical model rests on Combinatorics for counting favorable outcomes.

🔬 Where this appears

⏳ History

- 1654 — Blaise Pascal and Pierre de Fermat exchange letters on dividing stakes in an interrupted gambling game (the "problem of points"), founding mathematical probability.

- 1657 — Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed treatise on probability, introducing expectation.

- 1713 — Jacob Bernoulli's posthumous Ars Conjectandi states the Law of Large Numbers, linking probability to long-run frequency.

- 1718 — Abraham de Moivre's The Doctrine of Chances develops approximations leading toward the normal curve.

- 1763 — Thomas Bayes's theorem on conditional probability is published posthumously, founding Bayesian inference.

- 1812 — Pierre-Simon Laplace's Théorie analytique des probabilités systematizes the classical "equally likely" definition.

- 1900 — David Hilbert lists the axiomatization of probability as part of his sixth problem.

- 1933 — Andrey Kolmogorov publishes the axiomatic, measure-theoretic foundation that defines the modern probability model.

- 1950s onward — Probability becomes central to information theory (Shannon), statistical mechanics, and computer simulation (Monte Carlo methods at Los Alamos).

open-problem

The axioms guarantee a consistent model but never tell you which model is correct for a real phenomenon — this is the deep problem of interpreting probability. The frequentist view defines $P(A)$ as the limiting relative frequency over infinitely many repetitions, but no experiment is truly repeatable infinitely, and some events ("there is life on Europa") are not repeatable at all. The Bayesian view treats probability as a degree of belief, raising the question of where prior probabilities come from. On infinite or continuous sample spaces, not every subset can be assigned a probability — pathological "non-measurable" sets exist (related to the Banach–Tarski paradox), which is precisely why Kolmogorov restricted events to a σ-algebra. Choosing and justifying a model thus remains as much a matter of scientific and philosophical judgment as of mathematics.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also