Probability Spaces — Sigma-Algebras and Kolmogorov Axioms

⚡ Key Takeaways

A probability space is a triple $(\Omega, \mathcal{F}, P)$ where $\Omega$ is a sample space, $\mathcal{F}$ a $\sigma$ -algebra of measurable events, and $P$ a countably additive measure with $P(\Omega)=1$ . The Kolmogorov axioms (1933) are: non-negativity, normalization, and countable additivity. This is the rigorous measure-theoretic foundation of all modern probability, statistics, stochastic processes, and mathematical finance. Due to Andrey Kolmogorov, Grundbegriffe der Wahrscheinlichkeitsrechnung, 1933.

💡 The Core Insight

The genius of the Kolmogorov framework is separating "what can be measured" from "how it is measured." The $\sigma$ -algebra answers the former (it is a closure-under-countable-operations structure), and the measure answers the latter. Countable additivity — not mere finite additivity — is the precise amount of "continuity" needed to make limits, infinite sequences, and convergence theorems work, while still being weak enough to be realizable. Everything in probability is downstream of this one design choice.

DefinitionSigma-Algebra

Let $\Omega$ be a nonempty set. A collection $\mathcal{F}\subseteq 2^\Omega$ is a $\sigma$ -algebra (or $\sigma$ -field) on $\Omega$ if:

1

\quad \Omega \in \mathcal{F}

Derivation hint: The whole space is always an event — it is the certain event with probability 1.

2

\quad A \in \mathcal{F} \;\Longrightarrow\; A^c = \Omega\setminus A \in \mathcal{F}

Derivation hint: Closure under complementation; "not A" is an event whenever "A" is.

3

\quad A_1, A_2, A_3, \dots \in \mathcal{F} \;\Longrightarrow\; \bigcup_{n=1}^{\infty} A_n \in \mathcal{F}

Derivation hint: Closure under countable unions; the word " $\sigma$ " denotes "countable." Here $\Omega$ is the sample space, elements $\omega\in\Omega$ are outcomes, and members $A\in\mathcal{F}$ are events. From [1]–[3], $\mathcal{F}$ is automatically closed under countable intersections (via De Morgan) and contains $\varnothing$ .

DefinitionProbability Measure and the Kolmogorov Axioms

Given a measurable space $(\Omega,\mathcal{F})$ , a function $P:\mathcal{F}\to\mathbb{R}$ is a probability measure if it satisfies the three Kolmogorov axioms:

4

\quad P(A) \geq 0 \quad \text{for all } A \in \mathcal{F} \qquad \text{(non-negativity)}

Derivation hint: Probabilities measure "amount of belief/frequency" and cannot be negative.

5

\quad P(\Omega) = 1 \qquad \text{(normalization)}

Derivation hint: The certain event has total mass 1; this is the only axiom distinguishing probability from general measure theory.

6

\quad A_i \cap A_j = \varnothing \;(i\neq j)\;\Longrightarrow\; P\!\left(\bigcup_{n=1}^{\infty} A_n\right) = \sum_{n=1}^{\infty} P(A_n) \qquad \text{(countable additivity)}

Derivation hint: Disjoint events' probabilities add; the countable (not just finite) version is what enables limit theorems. The triple $(\Omega,\mathcal{F},P)$ is then a probability space.

TheoremElementary Consequences of the Axioms

From [1]–[6] the following hold for all $A,B\in\mathcal{F}$ :

7

\quad P(\varnothing)=0,\qquad P(A^c)=1-P(A),\qquad 0\le P(A)\le 1

Derivation hint: Apply additivity to $\Omega = A \sqcup A^c$ and use $P(\Omega)=1$ .

8

\quad A\subseteq B \;\Longrightarrow\; P(A)\le P(B)\quad\text{(monotonicity)}

Derivation hint: Write $B = A \sqcup (B\setminus A)$ and apply non-negativity.

9

\quad P(A\cup B)=P(A)+P(B)-P(A\cap B)\quad\text{(inclusion–exclusion)}

Derivation hint: Decompose $A\cup B$ into three disjoint pieces and add.

10

\quad P\!\left(\bigcup_{n=1}^\infty A_n\right)\le \sum_{n=1}^\infty P(A_n)\quad\text{(countable subadditivity / union bound)}

Derivation hint: Disjointify via $B_n = A_n\setminus\bigcup_{k<n}A_k$ , apply additivity then monotonicity.

TheoremContinuity of Probability Measures

Countable additivity is equivalent (given finite additivity) to either form of sequential continuity. For $A_n \uparrow A$ (increasing, $A=\bigcup A_n$ ) and $B_n \downarrow B$ (decreasing, $B=\bigcap B_n$ ):

11

\quad A_n\uparrow A \;\Longrightarrow\; P(A_n)\uparrow P(A); \qquad B_n\downarrow B \;\Longrightarrow\; P(B_n)\downarrow P(B)

Derivation hint: For the increasing case, write $A$ as a disjoint union of telescoping rings $A_n\setminus A_{n-1}$ and apply countable additivity.

ProofEquivalence of Countable Additivity and Continuity from Above at

\varnothing

We show: a finitely additive $P$ with $P(\Omega)=1$ is countably additive iff for every decreasing sequence $C_n\downarrow\varnothing$ we have $P(C_n)\to 0$ .

Step 1 (Additivity $\Rightarrow$ continuity). Let $C_n\downarrow\varnothing$ . Define disjoint rings $D_n=C_n\setminus C_{n+1}$ , so $C_1=\bigsqcup_{n\ge 1}D_n$ . By countable additivity $P(C_1)=\sum_{n\ge1}P(D_n)$ , a convergent series. The tail satisfies $P(C_m)=\sum_{n\ge m}P(D_n)\to 0$ as $m\to\infty$ since the series converges.

Step 2 (Continuity $\Rightarrow$ additivity). Let $\{A_n\}$ be disjoint with union $A=\bigsqcup_n A_n$ . Put $C_m = A\setminus\bigsqcup_{n=1}^{m}A_n = \bigsqcup_{n>m}A_n$ . Then $C_m\downarrow\varnothing$ , so $P(C_m)\to0$ . By finite additivity,

P(A)=\sum_{n=1}^{m}P(A_n)+P(C_m).

Step 3. Letting $m\to\infty$ and using $P(C_m)\to0$ gives $P(A)=\sum_{n=1}^\infty P(A_n)$ . $\qquad\blacksquare$

∎

ProofNecessity of a Sigma-Algebra: Why Not All Subsets?

Claim. No countably additive, translation-invariant $P$ exists on all of $2^{[0,1)}$ with $P([0,1))=1$ .

Step 1. Define $x\sim y$ iff $x-y\in\mathbb{Q}$ . By the axiom of choice, pick one representative from each equivalence class, forming the Vitali set $V\subseteq[0,1)$ .

Step 2. Enumerate $\mathbb{Q}\cap[0,1)=\{q_1,q_2,\dots\}$ and let $V_k = (V+q_k)\bmod 1$ . These are pairwise disjoint and $\bigsqcup_k V_k = [0,1)$ .

Step 3. By translation invariance all $P(V_k)$ are equal to some constant $c$ . Countable additivity gives $1=\sum_{k=1}^\infty c$ , which is $0$ if $c=0$ and $\infty$ if $c>0$ . Contradiction.

Conclusion. Hence $V$ cannot be assigned a consistent probability; one must restrict to a $\sigma$ -algebra (e.g. the Borel or Lebesgue $\sigma$ -algebra) that excludes such pathological sets. $\qquad\blacksquare$

∎

mathematician · Andrey Nikolaevich Kolmogorov | 1903–1987

Kolmogorov, working in Moscow, unified probability with Lebesgue's measure theory in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). Before him, probability was a loose collection of combinatorial rules and intuitions plagued by paradoxes (Bertrand's paradox, ill-defined "random points"). Kolmogorov's insight was that a random experiment is exactly a normalized measure space, and that conditional probability and independence are additional structures layered on top. He went on to found vast areas — turbulence theory, algorithmic complexity (Kolmogorov complexity), and the theory of stochastic processes (the Kolmogorov extension theorem, forward/backward equations).

🧠 Intuition

Think of $\Omega$ as the universe of "everything that could happen," and $P$ as one unit of sand spread over it. An event $A$ is a region; $P(A)$ is the amount of sand sitting on that region. The $\sigma$ -algebra $\mathcal{F}$ is the catalog of regions whose sand-content is well-defined — some regions (Vitali sets) are too jagged to weigh consistently, so we exclude them. Countable additivity says: if you partition a region into infinitely many disjoint sub-regions, the sand adds up exactly — no sand is "lost in the limit." This is precisely the property that fails for mere finite additivity, and it is the lifeblood of every convergence theorem: the law of large numbers, the central limit theorem, and martingale convergence all secretly depend on being able to take limits inside $P$ .

Figure

Architecture of a probability space: outcomes, events, and the measure

ExampleFinite Sample Space — A Fair Die

Given. Roll a fair six-sided die. $\Omega=\{1,2,3,4,5,6\}$ , $\mathcal{F}=2^\Omega$ (all $2^6=64$ subsets), and $P(\{\omega\})=\tfrac16$ for each $\omega$ .

Verify the axioms. Non-negativity [4] holds since each singleton has $P=\tfrac16>0$ . Normalization [5]: $P(\Omega)=\sum_{k=1}^6\tfrac16=1$ . Additivity [6] holds by construction since $\Omega$ is finite (countable additivity reduces to finite additivity here).

Compute. Let $A=\{$ even $\}=\{2,4,6\}$ and $B=\{>3\}=\{4,5,6\}$ .

$P(A)=\tfrac36=\tfrac12$ , $P(B)=\tfrac36=\tfrac12$ .
$A\cap B=\{4,6\}$ , so $P(A\cap B)=\tfrac26=\tfrac13$ .
By inclusion–exclusion [9]: $P(A\cup B)=\tfrac12+\tfrac12-\tfrac13=\tfrac23$ .

Check directly: $A\cup B=\{2,4,5,6\}$ , four outcomes, $P=\tfrac46=\tfrac23$ . ✓

\boxed{P(A\cup B)=\tfrac23 \approx 0.667\ \text{(dimensionless)}}

⚠️ Common mistake

Finite additivity is NOT enough. Many introductory treatments quietly assume "probabilities of disjoint events add," but only specify finite additivity. There exist genuinely different finitely-additive measures (e.g. those built from free ultrafilters / Banach limits) that are not countably additive, and for which the continuity property [11] and the limit theorems fail. Kolmogorov deliberately demanded countable additivity [6]. Conversely, do not over-claim: there is no consistent way to demand uncountable additivity — that would force $P([0,1])=0$ from $\sum_{x}P(\{x\})=\sum_x 0$ .

⚠️ Common mistake

"Event" $\neq$ "outcome," and " $P(A)=0$ " $\neq$ " $A$ is impossible." In a continuous space the event "the dart hits exactly $x=0.5$ " has probability 0 but is not impossible — it can occur. Likewise $P(A)=1$ does not mean $A=\Omega$ ; it means $A$ holds almost surely. The empty event $\varnothing$ is the only "truly impossible" event in the logical sense, but null events behave like it for measure purposes.

CounterexampleNon-measurable sets break naive probability

Suppose, contrary to the framework, that we insisted $\mathcal{F}=2^{[0,1)}$ and that $P$ were translation-invariant and countably additive with $P([0,1))=1$ . The Vitali set $V$ (proof above) yields $\bigsqcup_{k}(V+q_k)\bmod 1 = [0,1)$ with all pieces of equal probability $c$ , forcing $1=\sum_{k=1}^\infty c$ — impossible for any value of $c$ . Hence $V$ is non-measurable, and the $\sigma$ -algebra must be smaller than the full power set. This is not a technicality: it is the reason $\sigma$ -algebras exist.

connection

The probability space is the launching point for the whole edifice. A Random Variable is a measurable function $X:(\Omega,\mathcal{F})\to(\mathbb{R},\mathcal{B})$ , and its Probability Distribution is the pushforward measure $P\circ X^{-1}$ on $\mathbb{R}$ . Conditional Probability and Conditional Expectation are defined relative to sub- $\sigma$ -algebras $\mathcal{G}\subseteq\mathcal{F}$ , which encode "information." Independence (Probability) is a property of how $\sigma$ -algebras factorize. The entire apparatus of Measure Theory — Carathéodory extension, Dynkin's $\pi$ – $\lambda$ theorem, the monotone class theorem — supplies the existence and uniqueness machinery that guarantees these objects are well defined. The Kolmogorov Extension Theorem then lets us build infinite-dimensional spaces hosting Stochastic Processes.

🔬 Where this appears

⏳ History

- 1654 — Pascal–Fermat correspondence on the problem of points launches combinatorial probability.

- 1713 — Jacob Bernoulli's Ars Conjectandi states the (weak) law of large numbers for Bernoulli trials.

- 1812 — Laplace's Théorie analytique des probabilités codifies the classical "equally likely cases" definition.

- 1889 — Bertrand's paradox exposes the ambiguity of "random" without a specified measure.

- 1902–1904 — Lebesgue and Borel develop measure and integration theory, supplying the missing rigorous backbone.

- 1909 — Borel's strong law of large numbers and "normal numbers" first use measure-theoretic arguments in probability.

- 1923 — Wiener constructs the rigorous measure (Wiener measure) for Brownian motion on path space.

- 1933 — Kolmogorov's Grundbegriffe</strong> axiomatizes probability as measure theory, introducing the triple $(\Omega,\mathcal{F},P)$ , conditional expectation via $\sigma$ -algebras, and the extension theorem.

- 1930s–40s — Doob develops martingale theory and the modern theory of stochastic processes atop the Kolmogorov foundation.

Probability Spaces — Sigma-Algebras and Kolmogorov Axioms

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also