Stochastic Processes

⚡ Key Takeaways

Stochastic processes are collections of random variables indexed by time (or space), modeling systems that evolve under uncertainty. Key result: a process is fully characterized by its finite-dimensional distributions (Kolmogorov), and important classes — Markov chains, Poisson processes, Brownian motion — admit tractable analysis. Where it applies: finance, queueing, physics, biology, signal processing. Who/when: foundations laid by Markov (1906), Wiener (1923), and Kolmogorov (1933).

💡 The Core Insight

The entire behavior of a stochastic process is encoded not in any single $X_t$ but in how the $X_t$ at different times are jointly distributed. Two simplifying structures dominate the subject. Independence/stationarity says the statistical character does not depend on absolute time. The Markov property says the future depends on the past only through the present. Whenever one of these holds, the seemingly infinite-dimensional object collapses into something computable — transition probabilities, rate matrices, or a few moments — and that collapse is what makes the theory usable.

DefinitionStochastic Process

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space and $(E, \mathcal{E})$ a measurable state space. A stochastic process with index set $T$ is a family of random variables

1

\quad X = \{X_t : t \in T\}, \qquad X_t : \Omega \to E.

Derivation hint: each $X_t$ is $\mathcal{F}/\mathcal{E}$ -measurable; "process" just means the whole indexed family is given on one $(\Omega,\mathcal{F},\mathbb{P})$ .

$\Omega$ — sample space; an outcome $\omega \in \Omega$ .
$\mathcal{F}$ — $\sigma$ -algebra of events; $\mathbb{P}$ — probability measure.
$E$ — state space (e.g. $\mathbb{R}$ , $\mathbb{Z}$ , a finite set).
$T$ — index set: $T=\mathbb{N}$ (discrete time) or $T=[0,\infty)$ (continuous time).
For fixed $\omega$ , the map $t \mapsto X_t(\omega)$ is a sample path or trajectory.

DefinitionFinite-Dimensional Distributions

For any finite set of times $t_1 < \dots < t_n$ in $T$ , the finite-dimensional distribution (fdd) is the joint law

2

\quad \mu_{t_1,\dots,t_n}(A_1\times\cdots\times A_n) = \mathbb{P}(X_{t_1}\in A_1,\ \dots,\ X_{t_n}\in A_n).

Derivation hint: pull back the product event under the map $\omega \mapsto (X_{t_1}(\omega),\dots,X_{t_n}(\omega))$ and apply $\mathbb{P}$ . The fdds must be consistent: marginalizing one coordinate of $\mu_{t_1,\dots,t_n}$ gives the lower-order fdd, and permuting times permutes coordinates accordingly.

TheoremKolmogorov Extension Theorem

Given a consistent family of finite-dimensional distributions $\{\mu_{t_1,\dots,t_n}\}$ on a Polish state space, there exists a probability space $(\Omega,\mathcal{F},\mathbb{P})$ and a stochastic process $\{X_t\}_{t\in T}$ whose fdds are exactly the given family. Thus a process is its consistent fdds.

DefinitionMarkov Property

A process is a Markov process if for all $s<t$ and measurable $A$ ,

3

\quad \mathbb{P}(X_t \in A \mid \mathcal{F}_s) = \mathbb{P}(X_t \in A \mid X_s),

where $\mathcal{F}_s = \sigma(X_u : u\le s)$ is the history up to time $s$ . Derivation hint: condition on the full past $\mathcal F_s$ ; the Markov property asserts this equals conditioning on the single present value $X_s$ .

DefinitionStationarity

A process is (strictly) stationary if shifting time leaves all fdds unchanged: for every shift $h$ and times $t_1,\dots,t_n$ ,

4

\quad (X_{t_1+h},\dots,X_{t_n+h}) \stackrel{d}{=} (X_{t_1},\dots,X_{t_n}).

It is weakly (covariance) stationary if $\mathbb{E}[X_t]=\mu$ is constant and the autocovariance $\gamma(s,t)=\mathrm{Cov}(X_s,X_t)$ depends only on $|t-s|$ . Derivation hint: equate the joint distribution of any time-block with that of the same block translated by $h$ ; weak stationarity only constrains the first two moments.

ProofChapman–Kolmogorov Equation

Let $X$ be a discrete-time Markov chain on a countable state space with $n$ -step transition probabilities $p^{(n)}_{ij} = \mathbb{P}(X_n = j \mid X_0 = i)$ . We prove

5

\quad p^{(m+n)}_{ik} = \sum_{j} p^{(m)}_{ij}\, p^{(n)}_{jk}.

Derivation hint: condition on the state at the intermediate time $m$ and apply the Markov property to factor the joint probability.

Step 1. Decompose over the intermediate state $X_m=j$ using total probability:

p^{(m+n)}_{ik}=\mathbb{P}(X_{m+n}=k\mid X_0=i)=\sum_j \mathbb{P}(X_{m+n}=k,\ X_m=j\mid X_0=i).

Step 2. Factor each term with the multiplication rule:

=\sum_j \mathbb{P}(X_{m+n}=k\mid X_m=j,\ X_0=i)\,\mathbb{P}(X_m=j\mid X_0=i).

Step 3. Apply the Markov property: given $X_m=j$ , the future is independent of $X_0$ , so $\mathbb{P}(X_{m+n}=k\mid X_m=j, X_0=i)=p^{(n)}_{jk}$ .

Step 4. Substitute, recognizing $\mathbb{P}(X_m=j\mid X_0=i)=p^{(m)}_{ij}$ :

p^{(m+n)}_{ik}=\sum_j p^{(m)}_{ij}\,p^{(n)}_{jk}. \qquad \blacksquare

In matrix form with transition matrix $P=(p_{ij})$ , this is simply $P^{(m+n)}=P^{(m)}P^{(n)}$ , i.e. $P^{(n)}=P^n$ — the $n$ -step matrix is the $n$ -th power.

∎

TheoremStrong Law for Stationary Ergodic Processes (Birkhoff)

If $\{X_t\}$ is stationary and ergodic with $\mathbb{E}|X_0|<\infty$ , then the time average converges to the ensemble average almost surely:

6

\quad \frac{1}{n}\sum_{k=0}^{n-1} X_k \;\xrightarrow{\text{a.s.}}\; \mathbb{E}[X_0].

Derivation hint: Birkhoff's pointwise ergodic theorem applied to the shift map; ergodicity forces the invariant $\sigma$ -algebra to be trivial so the limit is the constant mean.

AK

Andrey Kolmogorov

1903–1987

Kolmogorov gave probability theory its modern measure-theoretic foundation in his 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung, where the extension theorem (Theorem above) showed that consistent fdds always yield a genuine process. He also derived the Kolmogorov forward and backward equations governing the time-evolution of transition densities for Markov processes, the bridge between probability and PDEs. His axioms turned an intuitive but shaky subject into rigorous mathematics, making the entire edifice of stochastic processes possible.

🧠 Intuition

Picture a particle on a number line. At each tick it flips a fair coin and steps $+1$ or $-1$ : this is the simple symmetric random walk, the discrete ancestor of everything. Now imagine making the time steps tinier and the spatial steps proportionally smaller; in the limit the jagged walk smooths into the continuous, infinitely-wiggly path of Brownian Motion. The Markov property is the statement that the particle "has no memory" — knowing its whole history tells you nothing more about its next step than knowing where it stands right now. Stationarity is the statement that the rules of the game never change, even though the particle's position does. These two ideas — memorylessness and time-homogeneity — are the levers that turn an impossibly rich object into something you can actually compute with.

Figure

Three sample paths of a symmetric random walk converging toward a Brownian envelope (dashed parabola

\pm\sqrt{t}

).

ExampleTwo-State Markov Chain — Long-Run Weather

Given: Weather is either Sunny (S) or Rainy (R). Transition matrix (rows = today, cols = tomorrow):

P=\begin{pmatrix} 0.8 & 0.2 \\ 0.4 & 0.6 \end{pmatrix}\quad\text{(S,R order)}.

Today is Sunny. Find (a) probability it's Rainy in 2 days, (b) the long-run fraction of rainy days.

Step 1 (two-step matrix). $P^2 = \begin{pmatrix}0.8&0.2\\0.4&0.6\end{pmatrix}^2$ . Top row:

$P^2_{SS}=0.8(0.8)+0.2(0.4)=0.64+0.08=0.72$
$P^2_{SR}=0.8(0.2)+0.2(0.6)=0.16+0.12=0.28$

So (a) $\mathbb{P}(\text{Rainy in 2 days}\mid\text{Sunny today}) = 0.28$ .

Step 2 (stationary distribution). Solve $\pi P = \pi$ , $\pi_S+\pi_R=1$ . From $\pi_S = 0.8\pi_S + 0.4\pi_R$ :

0.2\pi_S = 0.4\pi_R \Rightarrow \pi_S = 2\pi_R.

With $\pi_S+\pi_R=1$ : $3\pi_R=1$ , so $\pi_R=1/3$ , $\pi_S=2/3$ .

Answer:

\boxed{\ \mathbb{P}_2(\text{Rain})=0.28,\qquad \text{long-run rainy fraction}=\tfrac{1}{3}\approx 33.3\%\ }

ExamplePoisson Process — Customer Arrivals

Given: Customers arrive at a counter as a Poisson process with rate $\lambda = 5$ per hour. The number $N_t$ in $[0,t]$ satisfies $\mathbb{P}(N_t=k)=e^{-\lambda t}(\lambda t)^k/k!$ , and interarrival times are i.i.d. Exponential( $\lambda$ ).

Step 1 (probability of exactly 2 in 30 min). Here $t=0.5$ h, $\lambda t = 5(0.5)=2.5$ .

\mathbb{P}(N_{0.5}=2)=e^{-2.5}\frac{2.5^2}{2!}=e^{-2.5}\frac{6.25}{2}=0.0821\times 3.125=0.2565.

Step 2 (expected wait for next customer). Interarrival time $\sim$ Exp( $\lambda$ ), mean $1/\lambda = 1/5$ h $= 12$ min.

Step 3 (probability next customer takes > 20 min). $\mathbb{P}(T>1/3\,\text{h})=e^{-\lambda t}=e^{-5(1/3)}=e^{-1.667}=0.189.$

Answer:

\boxed{\ \mathbb{P}(\text{2 in 30 min})\approx 0.257,\quad \mathbb{E}[\text{wait}]=12\text{ min},\quad \mathbb{P}(\text{wait}>20\text{ min})\approx 0.189\ }

⚠️ The Markov property is not "no dependence on the past"

Markov processes can be highly correlated across time — yesterday's value strongly influences today's. The property only says that, conditional on the present, the past adds no further predictive information. People also wrongly think every process can be made Markov; in general you need to enlarge the state space (e.g. include velocity, or the last $k$ values) to recover the Markov property.

⚠️ Time average ≠ ensemble average in general

The equality "long-run time average = mean" (equation [6]) requires ergodicity, not merely stationarity. A stationary process can fail to be ergodic, in which case time averages converge to a random limit depending on which "regime" the system was born into.

CounterexampleA stationary but non-ergodic process

Let $B$ be a single fair coin flip determined once at time zero: $B\in\{0,1\}$ each with probability $1/2$ . Define $X_t = B$ for all $t$ . This process is strictly stationary (all fdds are shift-invariant). But the time average is $\frac1n\sum X_k = B$ , which equals $0$ or $1$ at random — never the ensemble mean $\mathbb{E}[X_0]=1/2$ . Ergodicity fails because the event " $B=1$ " is shift-invariant yet has probability $1/2$ , not $0$ or $1$ .

connection

Stochastic processes sit at a crossroads. Discrete-time chains link to Linear Algebra through transition-matrix eigenvalues that govern convergence to equilibrium. The continuous-time limit gives Brownian Motion, whose calculus is the Itô Calculus underpinning Stochastic Differential Equations. The evolution of transition densities is governed by the Kolmogorov Forward Equation (Fokker–Planck), connecting probability to Partial Differential Equations. Conditioning machinery rests on Conditional Expectation and the theory of Martingales, the processes with "fair-game" structure. Stationary processes connect to Fourier Analysis via the spectral density of the autocovariance function.

🔬 Where this appears

⏳ History

- 1827 — Robert Brown observes the erratic jittering of pollen grains in water (Brownian motion), with no mathematical model.

- 1900 — Louis Bachelier, in Théorie de la spéculation, models stock prices as Brownian motion — five years before Einstein, but largely ignored.

- 1905 — Albert Einstein gives the physical theory of Brownian motion, linking diffusion to molecular collisions and to Avogadro's number.

- 1906 — Andrey Markov introduces "Markov chains," analyzing dependent sequences (vowel/consonant patterns in Pushkin's poetry).

- 1923 — Norbert Wiener constructs Brownian motion rigorously as a measure on path space (the Wiener process).

- 1931–1933 — Kolmogorov derives the forward/backward equations and, in 1933, axiomatizes probability with the extension theorem.

- 1940s — Joseph Doob develops martingale theory and writes the foundational Stochastic Processes (1953).

- 1944–1951 — Kiyoshi Itô develops stochastic integration and Itô's lemma, founding Itô Calculus.

- 1960 — Rudolf Kálmán publishes the recursive filter for linear stochastic systems, revolutionizing engineering.

open-problem

Several deep questions and limitations persist. Path regularity: for a given consistent family of fdds, the Kolmogorov extension theorem guarantees a process but says nothing about continuity of paths; the Kolmogorov continuity theorem gives moment conditions for a continuous modification, but distinguishing versions/modifications is genuinely subtle. Non-Markovian memory: processes like fractional Brownian motion (with Hurst parameter $H\ne 1/2$ ) exhibit long-range dependence and are not Markov nor semimartingales, breaking standard Itô calculus; building a complete stochastic calculus for them remains active research. High-dimensional mixing rates: for Markov chains used in MCMC, proving sharp bounds on the mixing time (how fast $P^n$ reaches stationarity) is open for many practically important chains. Rough paths and SPDEs: giving rigorous meaning to stochastic PDEs driven by extremely irregular noise required Hairer's regularity-structures theory (Fields Medal 2014) and is far from fully understood.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also