Random Variables

⚡ Key Takeaways

A random variable is a measurable function that assigns a real number to each outcome of a random experiment. Key result in words: it transports probability from an abstract sample space onto the real line, where it is described by a distribution function, mass function, or density. Where it applies: all of probability theory, statistics, statistical physics, finance, information theory, and machine learning. Who/when: crystallized through Chebyshev and Kolmogorov (1933), building on Gauss, Laplace, and the measure theory of Borel and Lebesgue.

💡 The Core Insight

A random variable is the bridge between the messy, problem-specific world of outcomes and the clean, universal world of real numbers. Once you have $X$ , probability becomes the study of how mass is distributed along $\mathbb{R}$ . Everything — expectation, variance, independence, convergence — is a statement about that distribution. The sample space becomes scaffolding you can discard.

DefinitionProbability Space

A probability space is a triple $(\Omega, \mathcal{F}, P)$ where:

$\Omega$ is the sample space (set of all outcomes $\omega$ ),
$\mathcal{F}$ is a $\sigma$ -algebra of subsets of $\Omega$ (the events), closed under complement and countable union,
$P:\mathcal{F}\to[0,1]$ is a probability measure with $P(\Omega)=1$ and countable additivity.

The axioms of countable additivity read:

1

\quad P\!\left(\bigcup_{i=1}^{\infty} A_i\right) = \sum_{i=1}^{\infty} P(A_i), \qquad A_i \cap A_j = \varnothing \ (i\ne j).

Derivation hint: this is Kolmogorov's third axiom; it is an assumption, not a theorem, chosen to make limits of probabilities behave.

DefinitionRandom Variable

A random variable is a function $X:\Omega \to \mathbb{R}$ that is $\mathcal{F}$ -measurable: for every $x\in\mathbb{R}$ ,

2

\quad X^{-1}\big((-\infty, x]\big) = \{\omega\in\Omega : X(\omega)\le x\} \in \mathcal{F}.

Derivation hint: we require the preimage of every half-line to be an event so that $P(X\le x)$ makes sense; preimages of all Borel sets then follow because $\{(-\infty,x]\}$ generates the Borel $\sigma$ -algebra. Symbols: $\omega$ = an outcome; $X(\omega)$ = its numerical value; $X^{-1}(B)=\{\omega:X(\omega)\in B\}$ = preimage; $\mathcal{F}$ = the events for which probability is defined.

DefinitionDistribution and CDF

The cumulative distribution function (CDF) of $X$ is

3

\quad F_X(x) = P(X\le x) = P\big(\{\omega : X(\omega)\le x\}\big).

Derivation hint: apply $P$ to the measurable set guaranteed by equation [2]. $F_X$ is non-decreasing, right-continuous, with $\lim_{x\to-\infty}F_X(x)=0$ and $\lim_{x\to+\infty}F_X(x)=1$ . The law / distribution of $X$ is the pushforward measure $P_X(B)=P(X^{-1}(B))$ on the Borel sets.

DefinitionDiscrete vs Continuous

Discrete: $X$ takes values in a countable set; described by a probability mass function (PMF) $p_X(x)=P(X=x)$ with $\sum_x p_X(x)=1$ .

(Absolutely) continuous: there exists a probability density function (PDF) $f_X\ge 0$ with

4

\quad F_X(x) = \int_{-\infty}^{x} f_X(t)\,dt, \qquad \int_{-\infty}^{\infty} f_X(t)\,dt = 1.

Derivation hint: this is the Radon–Nikodym derivative of $P_X$ with respect to Lebesgue measure, when it exists; $f_X = F_X'$ wherever $F_X$ is differentiable.

TheoremExpectation and Variance

For an integrable random variable, the expectation is

5

\quad \mathbb{E}[X] = \int_{\Omega} X(\omega)\,dP(\omega) = \int_{-\infty}^{\infty} x \, dF_X(x),

Derivation hint: the abstract Lebesgue integral over $\Omega$ equals the Lebesgue–Stieltjes integral over $\mathbb{R}$ by the change-of-variables (pushforward) theorem. reducing to $\sum_x x\,p_X(x)$ (discrete) or $\int x f_X(x)\,dx$ (continuous). The variance is

6

\quad \operatorname{Var}(X) = \mathbb{E}\big[(X-\mathbb{E}[X])^2\big] = \mathbb{E}[X^2] - (\mathbb{E}[X])^2.

Derivation hint: expand $(X-\mu)^2 = X^2 - 2\mu X + \mu^2$ and use linearity of expectation with $\mu=\mathbb{E}[X]$ .

TheoremLaw of the Unconscious Statistician

For a measurable function $g:\mathbb{R}\to\mathbb{R}$ ,

7

\quad \mathbb{E}[g(X)] = \int_{-\infty}^{\infty} g(x)\,dF_X(x),

Derivation hint: $g(X)$ is itself a random variable; apply the pushforward of $P$ under $X$ , so no need to know the distribution of $g(X)$ explicitly.

ProofExpectation as a Distribution Integral

We justify equation [5]: that the integral over the abstract space $\Omega$ equals an integral over $\mathbb{R}$ using only the law $P_X$ . 1. Indicators. For $X = \mathbf{1}_A$ with $A\in\mathcal{F}$ , $\int_\Omega \mathbf{1}_A\,dP = P(A)$ . As a real-valued variable this $X$ takes value $1$ on $A$ and $0$ elsewhere, and $\int x\,dF_X = 0\cdot P(A^c) + 1\cdot P(A)=P(A)$ . The identity holds. 2. Simple functions. Let $X=\sum_{k=1}^n a_k \mathbf{1}_{A_k}$ with disjoint $A_k$ . By linearity of the integral, $\int_\Omega X\,dP = \sum_k a_k P(A_k)$ , which equals $\sum_k a_k\,P_X(\{a_k\}) = \int x\,dF_X$ . 3. Non-negative $X$ . Choose simple functions $X_n \uparrow X$ pointwise. By the Monotone Convergence Theorem, $\int_\Omega X_n\,dP \to \int_\Omega X\,dP$ , and correspondingly on $\mathbb{R}$ , giving the identity for non-negative $X$ . 4. General $X$ . Write $X = X^+ - X^-$ with $X^\pm \ge 0$ . If at least one of $\mathbb{E}[X^+], \mathbb{E}[X^-]$ is finite, subtract to obtain equation [5]. $\blacksquare$

∎

mathematician · Andrey Kolmogorov | 1903–1987

Kolmogorov's 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung placed probability on the rigorous footing of measure theory. His decisive move was to define a random variable as a measurable function on an abstract probability space, severing probability from any particular interpretation (frequency, betting, physical) and making it a branch of analysis. This let him prove the strong law of large numbers and the zero–one law with full rigor, and it remains the framework every undergraduate course silently assumes.

🧠 Intuition

Think of $\Omega$ as a bag of tickets, one per outcome $\omega$ , each printed with a number $X(\omega)$ . Drawing a ticket at random (weighted by $P$ ) and reading its number is the random variable. The measurability condition just says: for any threshold $x$ , the collection of tickets reading " $\le x$ " is one you're allowed to bet on (it lies in $\mathcal{F}$ ). Now imagine collapsing all tickets with the same number into one pile. The piles, with their total weights, are the distribution. Two utterly different bags (rolling a die vs. spinning a 6-sector wheel) can produce the same piles — hence the same random variable in distribution. This is why we routinely "forget $\Omega$ ": all probabilistic questions about $X$ alone are answered by the pile weights, i.e., by $F_X$ .

Figure

CDF and PDF of a continuous random variable: the CDF is the running area under the density.

```svg

```

ExampleA Fair Die — Discrete

Given: Roll a fair six-sided die. $\Omega=\{1,2,3,4,5,6\}$ , each outcome probability $1/6$ . Define $X(\omega)=\omega$ (the face value).

Steps: 1. PMF: $p_X(k)=1/6$ for $k=1,\dots,6$ . 2. Expectation, eqn [5]: $\mathbb{E}[X]=\sum_{k=1}^6 k\cdot\tfrac16 = \tfrac{1}{6}(1+2+3+4+5+6)=\tfrac{21}{6}=3.5$ . 3. $\mathbb{E}[X^2]=\tfrac16(1+4+9+16+25+36)=\tfrac{91}{6}\approx 15.1667$ . 4. Variance, eqn [6]: $\operatorname{Var}(X)=\tfrac{91}{6}-(3.5)^2 = 15.1667 - 12.25 = 2.9167$ .

Answer:

\boxed{\mathbb{E}[X]=3.5 \text{ (pips)}, \quad \operatorname{Var}(X)=\tfrac{35}{12}\approx 2.917 \text{ (pips}^2)}

⚠️ Common mistake

"A random variable is a number" — false. It is a function $X:\Omega\to\mathbb{R}$ . The number appears only after an outcome is drawn. Conflating the function with one of its values causes endless confusion when manipulating expressions like $\mathbb{E}[X]$ (a fixed number) versus $X$ (a function).

"Continuous means $P(X=x)>0$ for some $x$ " — false. For a continuous random variable $P(X=a)=0$ for every single point $a$ ; probability lives in intervals, not points. Density $f_X(a)$ is not a probability — it can exceed $1$ .

" $\mathbb{E}[g(X)]=g(\mathbb{E}[X])$ " — generally false (that is Jensen's inequality territory). Equality holds only for affine $g$ .

CounterexampleA Function That Is Not a Random Variable

Let $\Omega=[0,1]$ but take a coarse $\sigma$ -algebra $\mathcal{F}=\{\varnothing, [0,\tfrac12), [\tfrac12,1], \Omega\}$ . Define $X(\omega)=\omega$ . Then $\{X\le \tfrac14\}=[0,\tfrac14]$ , which is not in $\mathcal{F}$ . So $X$ fails the measurability condition [2] and is not a random variable with respect to this $\mathcal{F}$ — even though it is a perfectly ordinary function. Measurability is relative to the chosen $\sigma$ -algebra, not an intrinsic property of the formula.

connection

Random variables sit at the hub of probability. The notion of independence is defined via factorization of joint distributions of several random variables, leading to Joint Distributions. Sequences of them and their limiting behavior produce the Law of Large Numbers and Central Limit Theorem. The expectation operator is a special case of the Lebesgue Integration machinery, and the existence of densities invokes the Radon–Nikodym Theorem. The CDF connects to Measure Theory through Lebesgue–Stieltjes measures, and transformations $g(X)$ link to the Change of Variables formula. Conditioning generalizes to Conditional Expectation as a projection onto a sub- $\sigma$ -algebra.

🔬 Where this appears

⏳ History

- 1654 — Pascal and Fermat correspond on the "problem of points," computing expected stakes without naming random variables.

- 1713 — Jacob Bernoulli's Ars Conjectandi proves the first law of large numbers for binomial counts.

- 1733/1812 — De Moivre and Laplace develop the normal approximation; Laplace's Théorie analytique systematizes expectations and generating functions.

- 1867 — Chebyshev introduces the inequality bearing his name, treating moments of "variables" rigorously and proving a general LLN.

- 1901–1909 — Lyapunov and Markov sharpen the Central Limit Theorem using characteristic functions of sums of random variables.

- 1902–1904 — Lebesgue's integration theory provides the analytic engine for defining expectation as an integral.

- 1909–1933 — Borel and Fréchet develop measure on abstract spaces, paving the way for measurable functions.

- 1933 — Kolmogorov's Grundbegriffe gives the modern axiomatic definition: a random variable is a measurable function on a probability space.

- 1930s–40s — Doob and Kolmogorov extend the concept to stochastic processes (families of random variables indexed by time).

open-problem

Beyond the discrete/continuous dichotomy. Many distributions are neither purely discrete nor absolutely continuous. By the Lebesgue decomposition, every distribution splits into a discrete part, an absolutely continuous part, and a singular continuous part (e.g., the Cantor distribution), which has a continuous CDF but no density and assigns probability to a set of Lebesgue measure zero. Undergraduate courses usually ignore the singular part, yet it appears in dynamical systems and fractal models. Open practical questions: how to estimate and simulate from singular distributions efficiently, and how to define meaningful "densities" via Hausdorff measures. More foundationally, in infinite-dimensional settings (e.g., random functions, Gaussian processes) there is no Lebesgue measure, so the Radon–Nikodym viewpoint of equation [4] must be replaced by abstract measure theory — a frontier where measurability, the seemingly pedantic condition [2], becomes genuinely delicate.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also