Probability — Sample Spaces, Events, and Axioms

⚡ Key Takeaways

Probability theory begins with a triple $(\Omega, \mathcal{F}, P)$ : a sample space of outcomes, a $\sigma$ -algebra of events, and a probability measure satisfying three axioms (non-negativity, normalization, countable additivity). Every legitimate probabilistic statement is a theorem derivable from these axioms. Used everywhere quantitative uncertainty appears — statistics, physics, finance, machine learning, information theory. Axiomatized by Andrey Kolmogorov in 1933.

💡 The Core Insight

Probability is not about outcomes but about sets of outcomes. Once you accept that events form a $\sigma$ -algebra (a collection closed under complement and countable union) and that $P$ distributes additively over disjoint events, all of elementary probability follows as deductive consequences. The axioms are deliberately minimal: three short statements generate the entire edifice — the inclusion–exclusion principle, conditional probability, Bayes' theorem, the law of total probability, and continuity of measure.

DefinitionSample Space, Outcomes, Events

The sample space $\Omega$ is the non-empty set of all possible outcomes of an experiment. Each element $\omega \in \Omega$ is an outcome (or elementary event, or sample point). An event is a subset $A \subseteq \Omega$ that belongs to a designated collection $\mathcal{F}$ of "measurable" subsets. We say event $A$ occurs if the realized outcome $\omega$ satisfies $\omega \in A$ . Special events:

$\Omega$ — the certain event (always occurs).
$\varnothing$ — the impossible event (never occurs).
$A^c = \Omega \setminus A$ — the complement (" $A$ does not occur").
$A \cup B$ — " $A$ or $B$ "; $A \cap B$ — " $A$ and $B$ ".
$A, B$ are mutually exclusive (disjoint) iff $A \cap B = \varnothing$ .

Definition

\sigma

-algebra of Events

A collection $\mathcal{F}$ of subsets of $\Omega$ is a $\sigma$ -algebra (or $\sigma$ -field) if:

1

\quad \Omega \in \mathcal{F}

Derivation hint: the certain event must be measurable; this anchors normalization.

2

\quad A \in \mathcal{F} \;\Longrightarrow\; A^c \in \mathcal{F}

Derivation hint: closure under complement — if "A occurs" is an event, so is "A does not occur".

3

\quad A_1, A_2, A_3, \dots \in \mathcal{F} \;\Longrightarrow\; \bigcup_{i=1}^{\infty} A_i \in \mathcal{F}

Derivation hint: closure under countable union; by De Morgan, combined with [2] this gives closure under countable intersection. The pair $(\Omega, \mathcal{F})$ is called a measurable space. Elements of $\mathcal{F}$ are exactly the events.

DefinitionProbability Measure — Kolmogorov's Axioms

A function $P : \mathcal{F} \to \mathbb{R}$ is a probability measure if it satisfies the three Kolmogorov axioms:

Axiom I (Non-negativity).

4

\quad P(A) \ge 0 \qquad \text{for all } A \in \mathcal{F}

Derivation hint: probabilities are sizes of sets; sizes are non-negative.

Axiom II (Normalization).

5

\quad P(\Omega) = 1

Derivation hint: the certain event has full measure; total mass is normalized to one.

Axiom III (Countable Additivity / $\sigma$ -additivity). For any sequence of pairwise disjoint events $A_1, A_2, \dots$ (i.e. $A_i \cap A_j = \varnothing$ for $i \ne j$ ):

6

\quad P\!\left( \bigcup_{i=1}^{\infty} A_i \right) = \sum_{i=1}^{\infty} P(A_i)

Derivation hint: the probability of a disjoint union is the sum of the parts — additivity extended to countably many pieces, which is what enables limit/continuity arguments. The triple $(\Omega, \mathcal{F}, P)$ is a probability space.

TheoremElementary Consequences of the Axioms

From axioms [4]–[6] alone, for all events $A, B \in \mathcal{F}$ :

7

\quad P(\varnothing) = 0

Derivation hint: $\Omega = \Omega \cup \varnothing \cup \varnothing \cup \cdots$ disjointly; apply [6] and [5].

8

\quad P(A^c) = 1 - P(A)

Derivation hint: $A \cup A^c = \Omega$ disjointly; use [6] then [5].

9

\quad 0 \le P(A) \le 1

Derivation hint: combine [4] with [8] (since $P(A^c)\ge 0$ ).

10

\quad A \subseteq B \;\Longrightarrow\; P(A) \le P(B) \quad\text{(monotonicity)}

Derivation hint: write $B = A \cup (B \setminus A)$ disjointly; the extra term is non-negative.

11

\quad P(A \cup B) = P(A) + P(B) - P(A \cap B) \quad\text{(inclusion–exclusion)}

Derivation hint: decompose $A \cup B$ into three disjoint pieces $A\setminus B$ , $A\cap B$ , $B\setminus A$ and recombine.

12

\quad P\!\left(\bigcup_{i=1}^{\infty} A_i\right) \le \sum_{i=1}^{\infty} P(A_i) \quad\text{(countable subadditivity / union bound)}

Derivation hint: disjointify via $B_1=A_1$ , $B_k = A_k \setminus \bigcup_{j<k}A_j$ ; then $B_k \subseteq A_k$ and apply [6] and [10].

ProofProof of Finite Additivity and Inclusion–Exclusion

Step 1 — Finite additivity from [6]. Let $A_1, A_2$ be disjoint. Extend to an infinite sequence $A_1, A_2, \varnothing, \varnothing, \dots$ , which remains pairwise disjoint. By [6] and [7]:

P(A_1 \cup A_2) = P(A_1) + P(A_2) + \sum_{i \ge 3} P(\varnothing) = P(A_1) + P(A_2).

Step 2 — Decompose the union. For arbitrary $A, B$ , write the three disjoint pieces:

A\setminus B,\qquad A\cap B,\qquad B\setminus A.

These satisfy $A = (A\setminus B)\cup(A\cap B)$ , $B = (B\setminus A)\cup(A\cap B)$ , and $A\cup B = (A\setminus B)\cup(A\cap B)\cup(B\setminus A)$ , all unions disjoint.

Step 3 — Apply finite additivity.

P(A) = P(A\setminus B) + P(A\cap B), \qquad P(B) = P(B\setminus A) + P(A\cap B).

P(A\cup B) = P(A\setminus B) + P(A\cap B) + P(B\setminus A).

Step 4 — Eliminate the slice probabilities. Substitute $P(A\setminus B) = P(A) - P(A\cap B)$ and $P(B\setminus A) = P(B) - P(A\cap B)$ :

P(A\cup B) = \big[P(A) - P(A\cap B)\big] + P(A\cap B) + \big[P(B) - P(A\cap B)\big] = P(A) + P(B) - P(A\cap B). \qquad \blacksquare

∎

TheoremContinuity of Probability Measures

Let $A_1 \subseteq A_2 \subseteq \cdots$ be an increasing sequence of events with limit $A = \bigcup_n A_n$ . Then

13

\quad P(A) = \lim_{n\to\infty} P(A_n) = P\!\left(\bigcup_{n=1}^\infty A_n\right)

Derivation hint: disjointify into rings $B_n = A_n \setminus A_{n-1}$ (with $A_0=\varnothing$ ), apply [6], and recognize the partial sums as $P(A_n)$ . Dually, for a decreasing sequence $B_1 \supseteq B_2 \supseteq \cdots$ with $B = \bigcap_n B_n$ :

14

\quad P(B) = \lim_{n\to\infty} P(B_n)

Derivation hint: take complements to reduce to the increasing case [13], then apply [8]. These continuity properties are equivalent to countable additivity (given finite additivity), so they may be taken as an alternative form of Axiom III.

mathematician · Andrey Nikolaevich Kolmogorov | 1903–1987

The Soviet mathematician Kolmogorov gave probability its modern, measure-theoretic foundation in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). Before him, probability was a collection of techniques without a unifying axiomatic spine; rival foundations (frequency-based, à la von Mises) were heuristically motivated but logically fragile. Kolmogorov's stroke of genius was to recognize that Lebesgue's measure theory — developed for integration — was exactly the right language: a probability is simply a measure with total mass $1$ . This solved Hilbert's sixth problem (axiomatizing physics, including probability) for the probabilistic part, and made the law of large numbers, the central limit theorem, and stochastic processes rigorous objects. Kolmogorov also made foundational contributions to turbulence, complexity (Kolmogorov complexity), and dynamical systems.

🧠 Intuition

Think of $\Omega$ as a lump of clay of total mass $1$ . An event is a region you carve out, and $P(A)$ is the mass of that region. The axioms then read naturally:

Non-negativity: regions can't have negative mass.
Normalization: the whole lump weighs exactly $1$ .
Countable additivity: if you slice the clay into non-overlapping pieces, their masses add up — and this works even for infinitely many slices.

The complement rule $P(A^c) = 1 - P(A)$ is "mass outside the region equals total minus mass inside." Inclusion–exclusion corrects for the fact that, if two regions overlap, naively adding their masses double-counts the overlap $A\cap B$ , so you subtract it once. The union bound [12] says: smearing pieces around so they overlap can only increase apparent total mass, so the mass of the union never exceeds the sum. The reason we need a $\sigma$ -algebra rather than "all subsets" is that for a continuous lump (e.g. uniform clay on $[0,1]$ ) some pathological "regions" cannot be assigned a consistent mass at all — these are the non-measurable sets, and we simply refuse to call them events.

Figure

The probability space (Ω, 𝓕, P): outcomes, events, and overlapping mass

ExampleTwo Dice — Computing Event Probabilities

Given. Roll two fair, distinguishable six-sided dice. The sample space is

\Omega = \{(i,j) : i,j \in \{1,\dots,6\}\}, \qquad |\Omega| = 36,

with each outcome equally likely: $P(\{\omega\}) = 1/36$ .

Find. $P(A)$ where $A = \{$ sum $= 7\}$ , and $P(A \cup B)$ where $B = \{$ first die $= 1\}$ .

Step 1 — Event $A$ . Outcomes summing to $7$ : $(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)$ — that's $6$ outcomes.

P(A) = \frac{6}{36} = \frac{1}{6}.

Step 2 — Event $B$ . First die $= 1$ : $(1,1),\dots,(1,6)$ — $6$ outcomes.

P(B) = \frac{6}{36} = \frac{1}{6}.

Step 3 — Overlap $A \cap B$ . First die $1$ and sum $7$ : only $(1,6)$ . So $P(A\cap B) = 1/36$ .

Step 4 — Inclusion–exclusion [11].

P(A\cup B) = \frac{6}{36} + \frac{6}{36} - \frac{1}{36} = \frac{11}{36}.

Answer.

\boxed{P(A) = \tfrac{1}{6} \approx 0.167, \qquad P(A\cup B) = \tfrac{11}{36} \approx 0.306.}

ExampleContinuous Sample Space — A Uniform Dart

Given. A dart lands uniformly at random in the unit square $\Omega = [0,1]\times[0,1]$ . Here $\mathcal{F}$ is the Borel $\sigma$ -algebra and $P(A) = \text{area}(A)$ , which is a valid probability measure since the total area is $1$ .

Find. (a) $P(\{$ dart lands within distance $0.5$ of the origin $\})$ ; (b) $P(\{$ dart lands exactly on the diagonal $y=x\})$ .

Step 1 — Event (a). This is the quarter-disk of radius $0.5$ inside the square:

P = \text{area} = \frac{1}{4}\pi r^2 = \frac{1}{4}\pi (0.5)^2 = \frac{\pi}{16}.

Step 2 — Numerically.

P \approx \frac{3.14159}{16} \approx 0.1963.

Step 3 — Event (b). The diagonal $y=x$ is a line segment; as a subset of the plane it has area zero. Hence

P(\{y=x\}) = 0.

Key lesson. In a continuous space, probability zero does not mean impossible: the dart must land somewhere, every single point has probability zero, yet some point occurs. This is only consistent because uncountable unions are not covered by Axiom III.

Answer.

\boxed{(a)\; \frac{\pi}{16} \approx 0.196, \qquad (b)\; P = 0 \text{ (but not impossible).}}

⚠️ Common mistake

(1) "Every subset is an event." False for uncountable $\Omega$ . The power set of $[0,1]$ contains non-measurable sets (Vitali sets) to which no value $P$ can be consistently assigned while respecting translation-invariance and countable additivity. This is why $\mathcal{F}$ is required.

(2) "Probability $0$ means impossible; probability $1$ means certain." False in continuous spaces. Events of probability $0$ can occur ("almost never"), and events of probability $1$ can fail ("almost surely"). The correct phrasing is almost surely / almost never.

(3) "Additivity holds for any union." It holds only for disjoint events (Axiom III), and only for countably many of them. For overlapping events use inclusion–exclusion [11]; for uncountable unions there is no general additivity rule.

(4) " $P(A\cup B) = P(A) + P(B)$ always." Only when $A\cap B = \varnothing$ . Forgetting the overlap term is the single most common elementary error.

(5) "The sample space is unique." The same experiment can be modeled with different sample spaces depending on what is observed; only the induced probabilities of observable events must agree.

CounterexampleFinite Additivity Is Strictly Weaker Than Countable Additivity

Consider $\Omega = \mathbb{N} = \{1,2,3,\dots\}$ and suppose we demanded a "uniform" probability where every singleton has equal probability $p$ . If $p > 0$ , then countable additivity gives $P(\Omega) = \sum_{n} p = \infty \ne 1$ — contradiction. If $p = 0$ , then $P(\Omega) = \sum_n 0 = 0 \ne 1$ — contradiction. Conclusion: no uniform probability measure exists on a countably infinite set under Axiom III. It is possible to build a finitely-additive measure on $\mathbb{N}$ that assigns "density $0$ " to every finite set and is finitely additive — but it violates countable additivity. This shows that Axiom III (countable, not merely finite) is a genuine, non-trivial strengthening, and is exactly the ingredient that delivers continuity [13]–[14] and the limit theorems.

connection

The axioms are the launching point for the entire subject. Conditioning on an event $B$ with $P(B)>0$ defines Conditional Probability via $P(A\mid B) = P(A\cap B)/P(B)$ , which inverts to give Bayes' Theorem and the Law of Total Probability. Mappings $X:\Omega\to\mathbb{R}$ that are $\mathcal{F}$ -measurable are Random Variables, and the push-forward of $P$ along $X$ yields its Probability Distribution. Because $P$ is just a measure of total mass $1$ , all of Measure Theory and Lebesgue Integration applies, turning expectation into an integral $\mathbb{E}[X]=\int_\Omega X\, dP$ . The notion of Independence of events ( $P(A\cap B)=P(A)P(B)$ ) is a structural relation defined directly on the probability space, and indexed families of independent variables underlie the Law of Large Numbers and the Central Limit Theorem.

🔬 Where this appears

⏳ History

- 1654 — Blaise Pascal and Pierre de Fermat correspond about gambling problems (the "problem of points"), founding the systematic study of chance.

- 1657 — Christiaan Huygens publishes De ratiociniis in ludo aleae, the first printed treatise on probability, introducing expectation.

- 1713 — Jacob Bernoulli's posthumous Ars Conjectandi states the (weak) law of large numbers, linking long-run frequency to probability.

- 1763 — Thomas Bayes' essay (published posthumously by Richard Price) introduces inverse/conditional reasoning, later formalized as Bayes' theorem.

- 1812 — Pierre-Simon Laplace's Théorie analytique des probabilités systematizes the classical (equally-likely-outcomes) definition and generating-function methods.

- 1900 — David Hilbert's sixth problem calls for the axiomatization of physics, explicitly including probability.

- 1909–1919 — Émile Borel and others develop the connection between measure theory and probability; Richard von Mises proposes a frequency-based (collective) foundation.

- 1933 — Andrey Kolmogorov publishes Grundbegriffe der Wahrscheinlichkeitsrechnung, giving the now-standard measure-theoretic axioms

(\Omega,\mathcal{F},P)

— the definition used in this article.

- 1938–1956 — Doob, Lévy, and Kolmogorov build modern stochastic-process theory (martingales, Brownian motion) atop the axioms, vindicating the framework's generality.

open-problem

Foundational and philosophical frontiers.

The meaning of probability. Kolmogorov's axioms specify the mathematics but not the interpretation. The frequentist, Bayesian/subjectivist, and propensity interpretations all satisfy the same axioms yet disagree about what $P(A)$ means. There is no mathematical resolution; the choice affects statistical methodology.

Non-measurable sets and the Axiom of Choice. The existence of non-measurable subsets of $[0,1]$ depends on the Axiom of Choice. Under the Axiom of Determinacy (incompatible with full Choice), every set of reals is Lebesgue measurable. Thus whether "all subsets are events" is even consistent depends on one's set-theoretic axioms.

Finitely-additive probability. Some researchers (de Finetti) argued countable additivity is too strong and that only finite additivity is normatively justified, since the strong law of large numbers can fail behavior in finitely-additive settings. This remains a live debate in the foundations of statistics.

Conditioning on null events. $P(A\mid B)$ is undefined when $P(B)=0$ , yet such conditioning is routine (e.g., conditioning a continuous variable on $X=x$ ). Resolving this rigorously requires regular conditional probabilities and the Borel–Kolmogorov paradox shows the answer can depend on how the null event is approached — a genuine subtlety with no fully canonical fix.

Quantum probability. The non-distributive lattice of quantum events does not embed into a single classical Kolmogorov space (Bell inequalities), motivating generalized (non-Kolmogorovian) probability theories.

Probability — Sample Spaces, Events, and Axioms

Formal Statement

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also