Discrete Random Variables and Distributions

⚡ Key Takeaways

One-line: A discrete random variable maps outcomes of a random experiment to isolated numerical values, and its distribution assigns a probability to each value via a probability mass function (PMF).

Key result in words: The behaviour of any discrete random variable is completely determined by its PMF, which must be non-negative and sum to one; from it we compute every probability, expectation, variance, and moment.

Where it applies: Counting problems, success/failure trials, arrival counts, queueing, genetics, reliability, machine learning classification, and any setting with countably many distinguishable outcomes.

Who/when: Foundations laid by Pascal & Fermat (1654), Jacob Bernoulli (1713), Siméon Poisson (1837), with the modern measure-theoretic framework by Kolmogorov (1933).

💡 The Core Insight

A discrete random variable is fully captured by a list of (value, probability) pairs. Every question you can ask — "what is the chance $X$ exceeds 3?", "what is its average?", "how spread out is it?" — is just a weighted sum over that list. The PMF is the master object; everything else (CDF, mean, variance, moment generating function) is derived from it by summation.

DefinitionRandom Variable and Discreteness

Let $(\Omega, \mathcal{F}, P)$ be a probability space. A random variable is a measurable function $X : \Omega \to \mathbb{R}$ , meaning that for every real $x$ , the set $\{\omega \in \Omega : X(\omega) \le x\}$ belongs to $\mathcal{F}$ . $X$ is discrete if its image (range) is a finite or countably infinite set $\mathcal{X} = \{x_1, x_2, x_3, \dots\} \subset \mathbb{R}$ .

$\Omega$ — sample space (set of all outcomes)
$\mathcal{F}$ — $\sigma$ -algebra of events
$P$ — probability measure, $P:\mathcal{F}\to[0,1]$
$X(\omega)$ — the numerical value assigned to outcome $\omega$
$\mathcal{X}$ — the (countable) support of $X$

DefinitionProbability Mass Function

The probability mass function (PMF) of a discrete random variable $X$ is

1

\quad p_X(x) = P(X = x) = P\big(\{\omega : X(\omega) = x\}\big).

Derivation hint: this is just the probability measure $P$ evaluated on the event $\{X = x\}$ , the preimage $X^{-1}(\{x\})$ . A valid PMF satisfies two axioms:

2

\quad p_X(x) \ge 0 \ \ \forall x, \qquad \sum_{x \in \mathcal{X}} p_X(x) = 1.

Derivation hint: non-negativity comes from $P \ge 0$ ; the sum is $P(\Omega)=1$ because the events $\{X=x_i\}$ partition $\Omega$ (countable additivity). For any subset $A \subseteq \mathbb{R}$ ,

3

\quad P(X \in A) = \sum_{x \in A \cap \mathcal{X}} p_X(x).

Derivation hint: $\{X\in A\}$ is the disjoint union over $x\in A\cap\mathcal X$ of $\{X=x\}$ ; apply countable additivity.

DefinitionCumulative Distribution Function

The cumulative distribution function (CDF) is

4

\quad F_X(x) = P(X \le x) = \sum_{x_i \le x} p_X(x_i).

Derivation hint: $\{X\le x\}=\bigcup_{x_i\le x}\{X=x_i\}$ , a disjoint union; sum the masses. For a discrete variable, $F_X$ is a right-continuous, non-decreasing step function with jumps of size $p_X(x_i)$ at each $x_i$ . The PMF is recovered by

5

\quad p_X(x_i) = F_X(x_i) - F_X(x_i^-) = F_X(x_i) - \lim_{t\uparrow x_i} F_X(t).

Derivation hint: the jump at $x_i$ equals $P(X\le x_i)-P(X<x_i)=P(X=x_i)$ .

DefinitionExpectation, Variance, Moments

The expected value (mean) of $X$ , when the sum converges absolutely, is

6

\quad \mathbb{E}[X] = \mu = \sum_{x \in \mathcal{X}} x\, p_X(x).

Derivation hint: a probability-weighted average; it is the Lebesgue integral $\int X\,dP$ reduced to a sum for the discrete case. For a function $g$ , the Law of the Unconscious Statistician gives

7

\quad \mathbb{E}[g(X)] = \sum_{x \in \mathcal{X}} g(x)\, p_X(x).

Derivation hint: group outcomes by the value of $X$ ; one need not first find the PMF of $g(X)$ . The variance measures spread:

8

\quad \mathrm{Var}(X) = \sigma^2 = \mathbb{E}[(X-\mu)^2] = \mathbb{E}[X^2] - \mu^2.

Derivation hint: expand $(X-\mu)^2 = X^2 - 2\mu X + \mu^2$ and use linearity of $\mathbb{E}$ , with $\mathbb{E}[\mu]=\mu$ . The probability generating function (PGF), for $\mathbb{N}$ -valued $X$ , is

9

\quad G_X(s) = \mathbb{E}[s^X] = \sum_{k=0}^{\infty} s^k\, p_X(k), \qquad |s|\le 1.

Derivation hint: apply [7] with $g(x)=s^x$ ; derivatives at $s=1$ recover factorial moments.

TheoremCatalogue of Standard Discrete Distributions

The principal named families, with PMF, mean, and variance:

Bernoulli( $p$ ): $p_X(1)=p,\ p_X(0)=1-p$ ; $\ \mu=p,\ \sigma^2=p(1-p)$ .

Binomial( $n,p$ ):

10

\quad p_X(k)=\binom{n}{k}p^k(1-p)^{n-k},\quad k=0,\dots,n;\ \ \mu=np,\ \sigma^2=np(1-p).

Derivation hint: $X=\sum_{i=1}^n B_i$ of i.i.d. Bernoullis; the binomial coefficient counts arrangements of $k$ successes.

Geometric( $p$ ) (trials until first success, $k\ge 1$ ):

11

\quad p_X(k)=(1-p)^{k-1}p;\ \ \mu=\tfrac1p,\ \sigma^2=\tfrac{1-p}{p^2}.

Derivation hint: $k-1$ failures then a success; geometric series gives normalisation and mean.

Poisson( $\lambda$ ):

12

\quad p_X(k)=\frac{\lambda^k e^{-\lambda}}{k!},\quad k=0,1,2,\dots;\ \ \mu=\lambda,\ \sigma^2=\lambda.

Derivation hint: limit of Binomial( $n,\lambda/n$ ) as $n\to\infty$ ; normalisation uses the series $\sum \lambda^k/k! = e^{\lambda}$ .

ProofLinearity of Expectation and the Binomial Mean

We prove $\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]$ and use it for the binomial.

Step 1. Let $X,Y$ be discrete with joint PMF $p_{X,Y}(x,y)$ . By the LOTUS [7] applied to $g(x,y)=ax+by$ :

\mathbb{E}[aX+bY]=\sum_x\sum_y (ax+by)\,p_{X,Y}(x,y).

Step 2. Split the sum and factor constants:

= a\sum_x x\sum_y p_{X,Y}(x,y) + b\sum_y y\sum_x p_{X,Y}(x,y).

Step 3. The inner sums are the marginals: $\sum_y p_{X,Y}(x,y)=p_X(x)$ and $\sum_x p_{X,Y}(x,y)=p_Y(y)$ . Hence

= a\sum_x x\,p_X(x) + b\sum_y y\,p_Y(y) = a\mathbb{E}[X]+b\mathbb{E}[Y].

No independence was assumed.

Step 4 (Binomial mean). Write $X=\sum_{i=1}^n B_i$ where $B_i\sim\text{Bernoulli}(p)$ . Since $\mathbb{E}[B_i]=p$ , linearity gives $\mathbb{E}[X]=\sum_{i=1}^n p = np$ . $\blacksquare$

∎

ProofPoisson as a Limit of the Binomial

Step 1. Take Binomial( $n,p$ ) with $p=\lambda/n$ held so that $np=\lambda$ is fixed, and let $n\to\infty$ :

p_X(k)=\binom{n}{k}\Big(\frac{\lambda}{n}\Big)^k\Big(1-\frac{\lambda}{n}\Big)^{n-k}.

Step 2. Expand $\binom{n}{k}=\dfrac{n(n-1)\cdots(n-k+1)}{k!}$ and group:

=\frac{\lambda^k}{k!}\cdot\underbrace{\frac{n(n-1)\cdots(n-k+1)}{n^k}}_{\to\,1}\cdot\Big(1-\frac{\lambda}{n}\Big)^{n}\Big(1-\frac{\lambda}{n}\Big)^{-k}.

Step 3. As $n\to\infty$ : the falling-factorial ratio $\to 1$ , the last factor $\to 1$ , and $\big(1-\lambda/n\big)^n\to e^{-\lambda}$ .

Step 4. Therefore $p_X(k)\to \dfrac{\lambda^k e^{-\lambda}}{k!}$ , the Poisson PMF [12]. $\blacksquare$

∎

SD

Siméon Denis Poisson

1781–1840

A French mathematician and physicist, Poisson introduced the distribution now bearing his name in his 1837 treatise Recherches sur la probabilité des jugements en matière criminelle et en matière civile, where he studied the number of wrongful convictions. He framed it as the limit of rare events in many independent trials — exactly the "law of small numbers." Poisson also made deep contributions to electromagnetism, elasticity, and celestial mechanics; the distribution was something of an aside in his work, yet it became one of the most-used models in all of science, governing radioactive decay counts, telephone-call arrivals, and biological mutation rates.

🧠 Intuition

Picture the total probability as one kilogram of sand. A discrete distribution places the sand in distinct piles at separated points on the number line; a continuous distribution would spread it smoothly. The PMF tells you the weight of each pile. The CDF is what you get by walking left-to-right and accumulating sand — it stays flat between piles and jumps up by the pile's weight whenever you step on one, hence the staircase shape. The mean is the balance point of all the piles (a centre of mass). The variance is the average squared distance of sand from that balance point — small if the piles huddle near the mean, large if they sprawl. The Bernoulli is the atom: a single pile-pair at 0 and 1. The Binomial stacks $n$ such atoms; the Poisson is what the binomial becomes when each pile is tiny but there are very many — the "rare-events" regime.

ExampleQuality Control — Binomial

Given: A factory produces widgets, each defective independently with probability $p=0.05$ . A box holds $n=20$ widgets. Let $X$ = number of defectives.

Question: Find $P(X=2)$ , $\mathbb{E}[X]$ , $\mathrm{Var}(X)$ , and $P(X\ge 1)$ .

Steps: 1. $X\sim\text{Binomial}(20,0.05)$ . Using [10]:

P(X=2)=\binom{20}{2}(0.05)^2(0.95)^{18}=190\times 0.0025\times 0.3972 = 0.1887.

2. Mean: $\mathbb{E}[X]=np=20(0.05)=1.0$ defectives. 3. Variance: $\mathrm{Var}(X)=np(1-p)=20(0.05)(0.95)=0.95$ , so $\sigma\approx 0.975$ . 4. At least one defective: $P(X\ge 1)=1-P(X=0)=1-(0.95)^{20}=1-0.3585=0.6415.$

Answer:

\boxed{P(X=2)\approx 0.189,\quad \mu=1.0,\quad \sigma^2=0.95,\quad P(X\ge1)\approx 0.642}

ExampleServer Requests — Poisson

Given: A web server receives requests at an average rate of $\lambda=3$ per second. Let $X$ = number of requests in a one-second window, modelled as $X\sim\text{Poisson}(3)$ .

Question: Find $P(X=0)$ , $P(X\le 2)$ , and verify the variance.

Steps: 1. From [12]: $P(X=0)=\dfrac{3^0 e^{-3}}{0!}=e^{-3}=0.0498.$ 2. $P(X=1)=3e^{-3}=0.1494$ ; $\ P(X=2)=\dfrac{9}{2}e^{-3}=0.2240.$ 3. $P(X\le 2)=e^{-3}(1+3+4.5)=e^{-3}(8.5)=0.4232.$ 4. Variance equals the mean: $\mathrm{Var}(X)=\lambda=3$ , so $\sigma=\sqrt{3}\approx 1.732$ requests.

Answer:

\boxed{P(X=0)\approx 0.0498,\quad P(X\le2)\approx 0.423,\quad \sigma^2=3\ \text{requests}^2}

Figure

PMF (bars) and CDF (staircase) of a Binomial(10, 0.3)

⚠️ PMF values vs. probabilities; CDF endpoints

A PMF value is itself a probability, $p_X(x)=P(X=x)$ , so it must lie in $[0,1]$ . This differs from a continuous density, where the value $f(x)$ can exceed 1 and $P(X=x)=0$ .
For discrete variables, strict vs. non-strict inequalities matter: $P(X<3)\ne P(X\le 3)$ in general — they differ by $p_X(3)$ . Be especially careful when using $F_X$ .
The CDF is right-continuous with the value including the jump: $F_X(x)=P(X\le x)$ takes the top of the step, not the bottom.
$\mathbb{E}[g(X)]\ne g(\mathbb{E}[X])$ in general (Jensen's inequality); e.g. $\mathbb{E}[X^2]\ne(\mathbb{E}[X])^2$ unless $\mathrm{Var}(X)=0$ .

CounterexampleA mean that does not exist

Consider $X$ with PMF $p_X(k)=\dfrac{6}{\pi^2 k^2}$ for $k=1,2,3,\dots$ (valid since $\sum 1/k^2=\pi^2/6$ ). The expectation

\mathbb{E}[X]=\sum_{k=1}^{\infty} k\cdot\frac{6}{\pi^2 k^2}=\frac{6}{\pi^2}\sum_{k=1}^{\infty}\frac{1}{k}=\infty

diverges (harmonic series). So a perfectly legitimate discrete distribution can have no finite mean. The St. Petersburg game is the classic gambling instance: a finite, well-defined game with infinite expected payoff. Moral: "every distribution has a mean" is false; existence requires absolute convergence of [6].

connection

Discrete random variables sit inside the broader theory of Probability Measures and Sigma-Algebras, which make the word "measurable" in the definition precise. They contrast with Continuous Random Variables and Densities, where sums become integrals and PMFs become PDFs; the two are unified by the Lebesgue–Stieltjes Integral. Sums of independent discrete variables are handled by Convolution of Distributions and elegantly by the probability generating function [9]. The large- $n$ behaviour of binomial sums leads into the Central Limit Theorem (normal approximation) and the Law of Large Numbers. Conditional structure connects to Joint and Conditional Distributions and Bayes' Theorem.

🔬 Where this appears

⏳ History

- 1654 — Pascal and Fermat correspond on the problem of points, implicitly using binomial-coefficient counting; birth of probability.

- 1657 — Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed probability text, introducing expectation.

- 1713 — Jacob Bernoulli's posthumous Ars Conjectandi states the (weak) law of large numbers and systematises the binomial distribution.

- 1738 — Abraham de Moivre derives the normal approximation to the binomial, a precursor to the CLT.

- 1837 — Siméon Poisson introduces the Poisson distribution as a limit of rare binomial events.

- 1860s–1870s — Maxwell and Boltzmann use discrete counting in statistical mechanics; later the Bose–Einstein and Fermi–Dirac statistics.

- 1898 — Ladislaus Bortkiewicz publishes Das Gesetz der kleinen Zahlen, fitting the Poisson to deaths by horse-kick in the Prussian army — a famous empirical validation.

- 1933 — Andrey Kolmogorov's Grundbegriffe der Wahrscheinlichkeitsrechnung gives the axiomatic, measure-theoretic foundation, making "random variable" rigorous.

- 1940s–1950s — Feller's influential textbook An Introduction to Probability Theory and Its Applications canonises the modern treatment of discrete distributions.

open-problem · Boundaries, infinities, and modelling fidelity

Several genuine subtleties persist beyond the clean undergraduate picture. (1) Existence of moments: as the counterexample shows, means and variances may diverge for heavy-tailed discrete laws (e.g. Zipf/zeta distributions ubiquitous in linguistics and network science); choosing summary statistics then requires care. (2) Mixed and singular distributions: real data may combine a discrete atom (e.g. exactly-zero insurance claims) with a continuous part, requiring the Lebesgue–Stieltjes Integral for a unified treatment. (3) Overdispersion: empirical count data often have variance exceeding the mean, violating the Poisson's $\sigma^2=\mu$ identity and motivating the negative binomial — diagnosing the right model from finite data is an open practical challenge. (4) Infinite-support inference: estimating $\lambda$ or unseen species counts (the Good–Turing problem) from a finite sample remains an active area in statistics and information theory. (5) High-dimensional categorical data: modelling joint distributions over many discrete variables suffers the curse of dimensionality, driving research in graphical models and contrastive estimation.

Discrete Random Variables and Distributions

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also