Continuous Random Variables and Densities

⚡ Key Takeaways

A continuous random variable takes values in a continuum (an interval of $\mathbb{R}$ ) and is described not by point probabilities — which are all zero — but by a probability density function (PDF) $f(x)$ whose integral over a set gives the probability of landing in that set. Key result: $P(a \le X \le b) = \int_a^b f(x)\,dx$ , with $f(x)\ge 0$ and $\int_{-\infty}^\infty f(x)\,dx = 1$ . Applies wherever measurements vary smoothly — physics, statistics, engineering, finance. Formalized by Laplace, Gauss, and rigorously by Kolmogorov (1933).

💡 The Core Insight

For continuous variables, probability is area, not height. The density $f(x)$ answers "how concentrated is probability near $x$ ?" — formally $f(x)\,dx \approx P(x \le X \le x+dx)$ . You never read a probability off the curve directly; you always integrate. This single shift — from summing point masses (discrete case) to integrating a density (continuous case) — is the entire conceptual jump.

DefinitionContinuous Random Variable and Density

A random variable $X$ on a probability space $(\Omega,\mathcal{F},P)$ is (absolutely) continuous if there exists a nonnegative integrable function $f:\mathbb{R}\to[0,\infty)$ , the probability density function (PDF), such that for every $a\le b$ :

1

\quad P(a \le X \le b) = \int_a^b f(x)\,dx.

Derivation hint: This is the defining property; it asserts that the measure $P_X$ induced by $X$ is given by integrating $f$ against Lebesgue measure. The density must satisfy the two normalization axioms:

2

\quad f(x)\ge 0 \ \ \forall x, \qquad \int_{-\infty}^{\infty} f(x)\,dx = 1.

Derivation hint: Nonnegativity follows since probabilities are nonnegative; the total integral equals $P(-\infty<X<\infty)=P(\Omega)=1$ . The cumulative distribution function (CDF) is

3

\quad F(x) = P(X \le x) = \int_{-\infty}^{x} f(t)\,dt,

Derivation hint: Set $a\to-\infty$ , $b=x$ in [1]. and where $F$ is differentiable the density is recovered by the Fundamental Theorem of Calculus:

4

\quad f(x) = \frac{d}{dx}F(x) = F'(x).

Derivation hint: Differentiate [3]; FTC applies wherever $f$ is continuous at $x$ .

Symbols: $X$ = random variable; $\omega\in\Omega$ = sample point; $f$ = density (units: inverse of $X$ 's units); $F$ = CDF (dimensionless, in $[0,1]$ ); $x,t$ = real values.

TheoremPoint Probabilities Vanish

If $X$ is continuous with density $f$ , then for every $c\in\mathbb{R}$ ,

5

\quad P(X = c) = 0,

and consequently

6

\quad P(a\le X\le b) = P(a < X < b) = P(a\le X < b) = P(a< X\le b).

Derivation hint: $P(X=c)=\int_c^c f = 0$ by [1]; since endpoints contribute zero area, open/closed intervals share the same probability.

TheoremExpectation and Variance

For a continuous $X$ with density $f$ , and a measurable $g$ for which the integral converges absolutely (Law of the Unconscious Statistician):

7

\quad \mathbb{E}[g(X)] = \int_{-\infty}^{\infty} g(x)\,f(x)\,dx.

Derivation hint: Push the abstract expectation $\int_\Omega g(X)\,dP$ forward to the line via the distribution $P_X=f\,dx$ ; equivalently approximate $g$ by simple functions. In particular the mean $\mu$ and variance $\sigma^2$ are

8

\quad \mu = \mathbb{E}[X] = \int_{-\infty}^{\infty} x\,f(x)\,dx, \qquad \sigma^2 = \int_{-\infty}^{\infty} (x-\mu)^2 f(x)\,dx = \mathbb{E}[X^2]-\mu^2.

Derivation hint: Take $g(x)=x$ and $g(x)=(x-\mu)^2$ in [7]; expand the square and use linearity for the shortcut form.

ProofFrom Axioms to the Density Picture

We show that [1]–[4] are mutually consistent and that the change-of-variable formula for transformations follows.

Step 1 — CDF is nondecreasing and bounded. Since $f\ge 0$ , the integral $F(x)=\int_{-\infty}^x f$ is nondecreasing. As $x\to-\infty$ , $F\to 0$ ; as $x\to+\infty$ , $F\to\int_{-\infty}^\infty f = 1$ by [2]. Thus $F$ is a legitimate CDF.

Step 2 — Interval probabilities. By additivity of the integral,

P(a\le X\le b)=\int_a^b f = \int_{-\infty}^b f - \int_{-\infty}^a f = F(b)-F(a),

recovering [1] in terms of $F$ .

Step 3 — Recovering $f$ . Where $f$ is continuous, FTC gives $F'(x)=f(x)$ , which is [4]. Heuristically,

P(x\le X\le x+\Delta x)=F(x+\Delta x)-F(x)\approx f(x)\,\Delta x,

the "density times width" interpretation.

Step 4 — Transformation rule. Let $Y=u(X)$ with $u$ strictly monotonic and differentiable, inverse $x=u^{-1}(y)$ . For increasing $u$ , $F_Y(y)=P(u(X)\le y)=P(X\le u^{-1}(y))=F_X(u^{-1}(y))$ . Differentiating with the chain rule, and handling decreasing $u$ with an absolute value, yields the change-of-variables density

9

\quad f_Y(y) = f_X\!\big(u^{-1}(y)\big)\,\left|\frac{d}{dy}\,u^{-1}(y)\right|.

Derivation hint: Differentiate the CDF composition $F_Y=F_X\circ u^{-1}$ ; the absolute value compensates for orientation reversal when $u$ decreases. $\qquad\blacksquare$

∎

mathematician · Andrey Kolmogorov | 1903–1987

Kolmogorov's 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung placed probability on measure-theoretic foundations. In his framework a continuous random variable is a measurable function whose distribution is absolutely continuous with respect to Lebesgue measure; the density $f$ is precisely the Radon–Nikodym derivative $dP_X/d\lambda$ . This dissolved century-old ambiguities about "infinitely small probabilities" and explained why some continuous-CDF distributions (the Cantor distribution) have no density at all.

🧠 Intuition

Imagine spreading exactly one kilogram of fine sand along a number line. The density $f(x)$ is the mass per unit length at position $x$ — measured in kg/meter, not kg. To find how much sand lies between $a$ and $b$ , you cannot read it at a point (a point has zero width, hence zero sand); you must integrate the density over $[a,b]$ . The total sand is $1$ kg, matching $\int f = 1$ . A tall, narrow spike means sand piled densely (high probability nearby) but covering little width; the area of that spike is what counts. This is exactly why $f(x)$ can exceed $1$ : a Uniform $(0,\tfrac12)$ distribution has $f=2$ on its support — high density over a short interval still integrating to $1$ .

Figure

Density curve: probability is the shaded area, not the curve height

ExampleExponential Lifetime — Mean and a Tail Probability

Given: A component's lifetime $T$ (hours) is Exponential with rate $\lambda = 0.01\,\text{hr}^{-1}$ , density

f(t) = \lambda e^{-\lambda t},\quad t\ge 0,\quad f(t)=0,\ t<0.

Step 1 — Check normalization. $\int_0^\infty \lambda e^{-\lambda t}\,dt = [-e^{-\lambda t}]_0^\infty = 1.$ ✓

Step 2 — Mean lifetime via [8]:

\mu=\int_0^\infty t\,\lambda e^{-\lambda t}\,dt = \frac{1}{\lambda} = \frac{1}{0.01}.

(Integration by parts: $u=t,\ dv=\lambda e^{-\lambda t}dt$ .)

Step 3 — Probability it survives past 150 hr using the CDF $F(t)=1-e^{-\lambda t}$ :

P(T>150) = 1-F(150)=e^{-0.01\cdot 150}=e^{-1.5}.

\boxed{\mu = 100\ \text{hr}, \qquad P(T>150)=e^{-1.5}\approx 0.223}

ExampleTransforming a Uniform Variable

Given: $X\sim$ Uniform $(0,1)$ with $f_X(x)=1$ on $[0,1]$ . Define $Y = -\ln X$ . Find $f_Y(y)$ .

Step 1 — Invert. $y=-\ln x \Rightarrow x = e^{-y}$ , valid for $y>0$ (since $x\in(0,1)$ ).

Step 2 — Jacobian. $\dfrac{dx}{dy} = -e^{-y}$ , so $\left|\dfrac{dx}{dy}\right| = e^{-y}$ .

Step 3 — Apply [9].

f_Y(y) = f_X(e^{-y})\cdot e^{-y} = 1\cdot e^{-y} = e^{-y},\quad y>0.

Step 4 — Identify. This is Exponential with rate $\lambda=1$ .

\boxed{f_Y(y)=e^{-y}\ \ (y>0)\ \Longrightarrow\ Y\sim \text{Exponential}(1)}

This is the basis of inverse-transform sampling: applying $-\ln$ to uniform draws produces exponential samples.

⚠️ Common mistake

The density is not a probability. $f(x)$ has units of (probability per unit $x$ ) and may be arbitrarily large. Writing " $P(X=x)=f(x)$ " is wrong; the correct statement is $P(X=x)=0$ and $P(x\le X\le x+dx)\approx f(x)\,dx$ . Likewise, never claim " $f(x)\le 1$ " — only $\int f = 1$ and $f\ge 0$ are required.

⚠️ Common mistake

Probability zero ≠ impossible. Every single outcome of a continuous variable has probability $0$ , yet exactly one outcome occurs on each trial. "Almost surely" (probability $1$ ) is not the same as "always." This is why, when conditioning on a continuous variable taking a precise value, one must use densities and limits, not naive $P(A|B)=P(A\cap B)/P(B)$ (which gives $0/0$ ).

CounterexampleA Continuous CDF With No Density

The Cantor (devil's staircase) distribution has a CDF $F$ that is continuous everywhere and increases from $0$ to $1$ , so $P(X=c)=0$ for all $c$ — yet $F'(x)=0$ at almost every point and $F$ is not absolutely continuous. There is no density $f$ with $F(x)=\int_{-\infty}^x f$ . Thus "continuous CDF" is strictly weaker than "has a density." A density exists only for absolutely continuous distributions. This shows the chain [3]–[4] can fail without absolute continuity.

connection

Continuous variables sit opposite Discrete Random Variables, where probability mass functions replace densities and sums replace integrals; both are special cases of distributions described by a Cumulative Distribution Function. The density is rigorously the Radon–Nikodym Derivative of the distribution with respect to Lebesgue measure, tying the subject to Measure Theory. Expectation [7] is an instance of Lebesgue Integration. Two or more continuous variables are handled with Joint Probability Densities and marginals, and independence is characterized by factorization of joint densities. The transformation rule [9] generalizes to the multivariate Jacobian Change of Variables. Specific densities — the Normal Distribution, Exponential Distribution, and Uniform Distribution — are the workhorse examples.

🔬 Where this appears

⏳ History

- 1733 — Abraham de Moivre derives the bell-curve approximation to the binomial, the first appearance of the normal density.

- 1774 — Pierre-Simon Laplace introduces continuous error distributions and integrates densities to assess astronomical errors.

- 1809 — Carl Friedrich Gauss formalizes the normal density in the context of least-squares estimation.

- 1812 — Laplace's Théorie analytique des probabilités systematizes integral-based probability and characteristic functions.

- 1837 — Siméon Poisson and contemporaries develop the analytic treatment of continuous limits.

- 1898 — Émile Borel develops measure theory on the line, the geometric backbone for "area = probability."

- 1902 — Henri Lebesgue's integral provides the rigorous tool for [7] and unbounded densities.

- 1913 — Radon (later Nikodym, 1930) prove the Radon–Nikodym theorem, defining the density as a derivative of measures.

- 1933 — Andrey Kolmogorov's axioms unify discrete and continuous probability under measure theory, the modern standard.

open-problem

The density is determined only up to a set of Lebesgue measure zero — two densities differing at countably many points describe the same distribution, so "the" density is really an equivalence class. Distributions can be purely continuous yet singular (Cantor type), having no density at all; the Lebesgue decomposition splits any distribution into absolutely continuous, discrete, and singular parts, and characterizing the singular component of empirical distributions remains subtle. In infinite dimensions (e.g. Gaussian measures on function spaces), there is no Lebesgue reference measure, so densities must be taken relative to another measure (Cameron–Martin theory) — a live area in stochastic analysis. Finally, density estimation from finite data (choosing bandwidths, avoiding the curse of dimensionality) is an active statistical and machine-learning research frontier.

Continuous Random Variables and Densities

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also