Random Variables and Characteristic Functions

⚡ Key Takeaways

The characteristic function of a random variable $X$ is $\varphi_X(t) = \mathbb{E}[e^{itX}]$ — the Fourier transform of its probability distribution. Key result: the characteristic function always exists, uniquely determines the distribution, and converts convolution into multiplication, making it the master tool for sums of independent variables, moment extraction, and proofs of the Central Limit Theorem. Applies wherever probability distributions, sums of random effects, or limit theorems appear. Introduced by Lyapunov and Lévy (early 20th century), built on Fourier (1820s) and refined by the Russian school.

💡 The Core Insight

Multiplication is easier than convolution. Adding independent random variables convolves their densities — a messy integral — but multiplies their characteristic functions. By translating probability into the language of complex exponentials, hard analytic operations become arithmetic, and the unique correspondence between $\varphi_X$ and the distribution guarantees nothing is lost in translation.

DefinitionRandom Variable

Given a probability space $(\Omega,\mathcal{F},\mathbb{P})$ , a random variable is a measurable function $X:\Omega\to\mathbb{R}$ , meaning $\{\omega : X(\omega)\le x\}\in\mathcal{F}$ for every $x\in\mathbb{R}$ . Its cumulative distribution function (CDF) is

1

\quad F_X(x) = \mathbb{P}(X \le x) = \mathbb{P}(\{\omega : X(\omega)\le x\}).

Derivation hint: directly evaluate the probability measure on the preimage of $(-\infty,x]$ ; measurability guarantees this set lies in $\mathcal{F}$ . If $F_X$ is absolutely continuous, there is a probability density function (PDF) $f_X$ with $F_X(x)=\int_{-\infty}^x f_X(u)\,du$ . The expectation of a function $g(X)$ is

2

\quad \mathbb{E}[g(X)] = \int_{-\infty}^{\infty} g(x)\,dF_X(x) = \int_{-\infty}^{\infty} g(x)\,f_X(x)\,dx \quad(\text{continuous case}).

Derivation hint: the law of the unconscious statistician — integrate $g$ against the pushforward measure $\mathbb{P}\circ X^{-1}$ , written as a Lebesgue–Stieltjes integral against $F_X$ .

DefinitionCharacteristic Function

The characteristic function (CF) of a random variable $X$ is the function $\varphi_X:\mathbb{R}\to\mathbb{C}$ defined by

3

\quad \varphi_X(t) = \mathbb{E}\!\left[e^{itX}\right] = \int_{-\infty}^{\infty} e^{itx}\, dF_X(x),

Derivation hint: apply [2] with $g(x)=e^{itx}=\cos tx + i\sin tx$ , splitting into real and imaginary integrable parts since $|\cos|,|\sin|\le 1$ . where $t\in\mathbb{R}$ and $i=\sqrt{-1}$ . Symbols: $\varphi_X$ is complex-valued; $t$ is a real parameter (the "frequency"); $dF_X$ denotes integration against the distribution (sum over a probability mass function in the discrete case).

TheoremFundamental Properties

For any random variable $X$ :

4

\quad \varphi_X(0)=1,\qquad |\varphi_X(t)|\le 1,\qquad \overline{\varphi_X(t)}=\varphi_X(-t),

Derivation hint: $\varphi_X(0)=\mathbb{E}[1]=1$ ; $|\varphi_X(t)|=|\mathbb{E}[e^{itX}]|\le\mathbb{E}|e^{itX}|=1$ by Jensen/triangle inequality; conjugate $e^{itx}$ . and $\varphi_X$ is uniformly continuous on $\mathbb{R}$ . For an affine transform $Y=aX+b$ ,

5

\quad \varphi_{aX+b}(t) = e^{itb}\,\varphi_X(at).

Derivation hint: $\mathbb{E}[e^{it(aX+b)}]=e^{itb}\,\mathbb{E}[e^{i(ta)X}]$ , factoring the deterministic exponential out of the expectation.

TheoremConvolution / Sum of Independents

If $X$ and $Y$ are independent random variables, then for $S=X+Y$ :

6

\quad \varphi_{X+Y}(t) = \varphi_X(t)\,\varphi_Y(t).

Derivation hint: $\mathbb{E}[e^{it(X+Y)}]=\mathbb{E}[e^{itX}e^{itY}]=\mathbb{E}[e^{itX}]\,\mathbb{E}[e^{itY}]$ , where independence factors the expectation of the product.

TheoremMoment Generation

If $\mathbb{E}|X|^n<\infty$ , then $\varphi_X$ is $n$ -times differentiable and

7

\quad \mathbb{E}[X^n] = \frac{1}{i^n}\,\varphi_X^{(n)}(0) = (-i)^n\,\varphi_X^{(n)}(0).

Derivation hint: differentiate under the integral sign $n$ times, bringing down a factor $(ix)^n$ , then set $t=0$ ; integrability of $|X|^n$ justifies the interchange via dominated convergence.

TheoremInversion and Uniqueness

The characteristic function determines the distribution uniquely: if $\varphi_X=\varphi_Y$ then $F_X=F_Y$ . When $\varphi_X\in L^1(\mathbb{R})$ , the density exists and is recovered by

8

\quad f_X(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-itx}\,\varphi_X(t)\,dt.

Derivation hint: this is the inverse Fourier transform; substitute $\varphi_X(t)=\int e^{ity}f_X(y)\,dy$ and use $\frac{1}{2\pi}\int e^{it(y-x)}\,dt=\delta(y-x)$ as a distributional identity (rigorously via the Lévy inversion formula with a Gaussian damping limit).

ProofUniqueness via Lévy's Inversion Formula

We show $\varphi$ determines $F$ at continuity points.

Step 1. For real $a<b$ , consider the integral

I_T=\frac{1}{2\pi}\int_{-T}^{T}\frac{e^{-ita}-e^{-itb}}{it}\,\varphi_X(t)\,dt.

Step 2. Insert $\varphi_X(t)=\int e^{itx}\,dF_X(x)$ and exchange the order of integration (justified by Fubini since the integrand is bounded on $[-T,T]\times\mathbb{R}$ ):

I_T=\int_{-\infty}^{\infty}\left[\frac{1}{2\pi}\int_{-T}^{T}\frac{e^{it(x-a)}-e^{it(x-b)}}{it}\,dt\right]dF_X(x).

Step 3. The inner bracket is a Dirichlet-type kernel; using $\int_{-T}^{T}\frac{\sin(\theta t)}{t}\,dt\to\pi\,\mathrm{sgn}(\theta)$ as $T\to\infty$ , the bracket converges to $\tfrac12[\mathbf{1}_{x>a}-\mathbf{1}_{x>b}]$ plus boundary terms.

Step 4. Taking $T\to\infty$ yields

\lim_{T\to\infty} I_T = F_X(b)-F_X(a)

at continuity points $a,b$ . Hence the increments of $F_X$ — and so $F_X$ itself — are fully determined by $\varphi_X$ . $\blacksquare$

∎

mathematician · Paul Lévy | 1886–1971

The French mathematician Paul Lévy was the architect of the modern theory of characteristic functions. In his 1925 Calcul des probabilités he proved the inversion formula bearing his name and the continuity theorem linking pointwise convergence of characteristic functions to convergence in distribution — the engine of nearly every modern proof of the Central Limit Theorem. Lévy also pioneered stochastic processes (Lévy processes, Brownian motion fine structure), stable laws, and the concept of infinitely divisible distributions, for which characteristic functions are the natural language via the Lévy–Khintchine formula.

🧠 Intuition

Think of a distribution as a "shape" of mass spread along the real line. The characteristic function is its fingerprint in frequency space: $\varphi_X(t)$ measures how much the distribution "resonates" with a wave $e^{itx}$ of frequency $t$ . At $t=0$ all the mass is counted in phase, giving $\varphi(0)=1$ . As $t$ grows, the wave oscillates faster; if the distribution is smooth and concentrated, contributions from nearby mass start cancelling, so $\varphi$ shrinks — narrow distributions have wide fingerprints and vice versa (the uncertainty principle of Fourier analysis). The magic of $|e^{itX}|=1$ is that no matter how heavy the tails — even for a Cauchy variable with no mean — the wave never blows up, so the fingerprint always exists. Sums of independent variables stack their wave-distortions multiplicatively, which is why adding randomness is a multiplication in fingerprint space.

ExampleSum of Two Independent Poissons

Given: $X\sim\text{Poisson}(\lambda_1)$ and $Y\sim\text{Poisson}(\lambda_2)$ , independent, with $\lambda_1=3$ , $\lambda_2=5$ . Find the distribution of $S=X+Y$ .

Step 1 — CF of a Poisson. For $X\sim\text{Poisson}(\lambda)$ :

\varphi_X(t)=\sum_{k=0}^{\infty}e^{itk}\frac{e^{-\lambda}\lambda^k}{k!}=e^{-\lambda}\sum_{k=0}^\infty\frac{(\lambda e^{it})^k}{k!}=e^{\lambda(e^{it}-1)}.

Step 2 — Multiply (independence, [6]).

\varphi_S(t)=e^{\lambda_1(e^{it}-1)}\cdot e^{\lambda_2(e^{it}-1)}=e^{(\lambda_1+\lambda_2)(e^{it}-1)}.

Step 3 — Recognize the form. This is exactly the CF of a Poisson with parameter $\lambda_1+\lambda_2=8$ . By uniqueness ([8]),

\boxed{S\sim\text{Poisson}(8),\qquad \mathbb{P}(S=k)=\frac{e^{-8}\,8^k}{k!}}

The Poisson family is closed under independent addition — proven in three lines with characteristic functions instead of a convolution sum.

ExampleMean and Variance of the Exponential via Differentiation

Given: $X\sim\text{Exponential}(\lambda)$ with density $f_X(x)=\lambda e^{-\lambda x}$ , $x\ge0$ , and $\lambda=2$ . Find $\mathbb{E}[X]$ and $\mathrm{Var}(X)$ .

Step 1 — Characteristic function.

\varphi_X(t)=\int_0^\infty e^{itx}\lambda e^{-\lambda x}\,dx=\lambda\int_0^\infty e^{-(\lambda-it)x}\,dx=\frac{\lambda}{\lambda-it}.

Step 2 — First derivative ([7], $n=1$ ).

\varphi_X'(t)=\frac{\lambda\, i}{(\lambda-it)^2},\qquad \varphi_X'(0)=\frac{\lambda i}{\lambda^2}=\frac{i}{\lambda}.

So $\mathbb{E}[X]=\frac{1}{i}\varphi_X'(0)=\frac{1}{\lambda}=\frac{1}{2}=\mathbf{0.5}$ .

Step 3 — Second derivative.

\varphi_X''(t)=\frac{-2\lambda}{(\lambda-it)^3},\qquad \varphi_X''(0)=\frac{-2\lambda}{\lambda^3}=\frac{-2}{\lambda^2}.

So $\mathbb{E}[X^2]=(-i)^2\varphi_X''(0)=-\,\varphi_X''(0)=\frac{2}{\lambda^2}=\frac{2}{4}=0.5$ .

Step 4 — Variance.

\mathrm{Var}(X)=\mathbb{E}[X^2]-(\mathbb{E}[X])^2=\frac{2}{\lambda^2}-\frac{1}{\lambda^2}=\frac{1}{\lambda^2}.

\boxed{\mathbb{E}[X]=0.5,\qquad \mathrm{Var}(X)=0.25}$$ (in the units of $X$; variance in squared units).

⚠️ Common mistake

The characteristic function is not the moment generating function (MGF). The MGF is $M_X(t)=\mathbb{E}[e^{tX}]$ (no $i$ ), and it can fail to exist for heavy-tailed distributions because $e^{tX}$ may have infinite expectation. The characteristic function $\mathbb{E}[e^{itX}]$ always exists because $|e^{itX}|=1$ . When the MGF exists in a neighborhood of $0$ , the two are related by $\varphi_X(t)=M_X(it)$ . Never assume an MGF exists; the CF is the safe universal tool.

⚠️ Common mistake

A real, symmetric distribution has a real-valued CF — but the converse logic is subtle. $\varphi_X$ is real-valued iff $X$ is symmetric about $0$ (i.e. $X$ and $-X$ have the same distribution). Students often forget that a general $\varphi_X(t)$ is genuinely complex. Also, $\varphi_X$ being differentiable at $0$ does not guarantee a finite mean unless you have the stronger two-sided condition; differentiability of $\varphi$ and existence of moments must be handled carefully.

CounterexampleThe Cauchy distribution has no mean — and its CF reveals why

The standard Cauchy distribution has density $f(x)=\frac{1}{\pi(1+x^2)}$ and characteristic function

\varphi_X(t)=e^{-|t|}.

This function is continuous and $|\varphi_X|\le1$ as required, but it is not differentiable at $t=0$ (the left and right derivatives are $+1$ and $-1$ ). By [7], this non-differentiability signals that $\mathbb{E}[X]$ does not exist. Indeed $\int|x|f(x)\,dx=\infty$ . Moreover, if $X_1,\dots,X_n$ are iid Cauchy, then $\varphi_{\bar X}(t)=\left(e^{-|t/n|}\right)^n=e^{-|t|}$ — the sample mean of $n$ Cauchy variables is again standard Cauchy, so averaging never reduces the spread. The Law of Large Numbers fails, and the CF makes this transparent.

connection

Characteristic functions sit at the crossroads of analysis and probability. They are the Fourier Transform of the distribution, inheriting inversion, convolution, and Plancherel theorems wholesale. Pointwise convergence of $\varphi_n\to\varphi$ (with $\varphi$ continuous at $0$ ) is equivalent to convergence in distribution by Lévy's Continuity Theorem, which is the standard route to the Central Limit Theorem. The class of CFs that are limits of CFs of sums forms the infinitely divisible distributions, classified by the Lévy–Khintchine Formula. For random vectors, the multivariate CF $\varphi_{\mathbf X}(\mathbf t)=\mathbb{E}[e^{i\mathbf t\cdot\mathbf X}]$ encodes the joint law and detects independence. The cousin generating tools — Moment Generating Function and Probability Generating Function — are special evaluations valid under stronger conditions.

🔬 Where this appears

⏳ History

- 1733–1812 — Foundations of the normal law. De Moivre, Laplace, and Gauss develop the bell curve; Laplace uses transform methods (the "generating function") foreshadowing characteristic functions.

- 1822 — Fourier. Joseph Fourier publishes the Théorie analytique de la chaleur, establishing the Fourier transform that characteristic functions specialize.

- 1900s — Lyapunov. Aleksandr Lyapunov gives a rigorous proof of the Central Limit Theorem using characteristic-function (transform) techniques, the first modern use in probability.

- 1920s — Lévy. Paul Lévy systematizes characteristic functions in Calcul des probabilités (1925), proving the inversion formula and continuity theorem.

- 1933 — Kolmogorov. Kolmogorov's measure-theoretic axiomatization of probability gives random variables and their distributions a rigorous foundation, legitimizing

\mathbb{E}[e^{itX}]

.

- 1934–1937 — Khintchine & Lévy. The Lévy–Khintchine formula classifies infinitely divisible distributions via their characteristic functions.

- 1936 — Bochner's theorem. Salomon Bochner characterizes exactly which functions are characteristic functions (continuous, positive-definite, value 1 at 0).

- 1937 — Cramér. Harald Cramér's Random Variables and Probability Distributions consolidates the theory for a generation of statisticians.

- 1990s — Computational era. Carr–Madan (1999) and the rise of FFT-based inversion make characteristic functions a practical engine in quantitative finance and computational statistics.

open-problem

Several deep questions surround characteristic functions. Bochner's theorem tells us which functions are CFs (continuous, positive-definite, $\varphi(0)=1$ ), but verifying positive-definiteness for a candidate function is often hard; constructing distributions with prescribed properties via their CFs remains delicate. The relationship between smoothness of $\varphi$ and existence of moments is only partly characterized: $2n$ -fold differentiability of $\varphi$ at $0$ implies finite $2n$ -th moments, but the odd-order and one-sided cases hide pathologies. Decay rates of $\varphi$ at infinity govern the smoothness of the density (a faster-decaying CF means a smoother density), tying into open questions in harmonic analysis about which decay/regularity pairings are achievable. Finally, in high dimensions and for dependent structures, recovering and estimating multivariate distributions from empirical characteristic functions faces curse-of-dimensionality and stability issues that remain active research in modern statistics.

Random Variables and Characteristic Functions

Formal Statement

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also