Grinstead-Snell-Probability · Problem 11.2.27

Markov Chains

Mathematics · absorbing-markov-chains

Problem

Consider an absorbing Markov chain with state space

S

. Let

f

be a function defined on

S

with the property that

f(i) = \sum_{j \in S} p_{ij} f(j),

or in vector form

\mathbf{f} = \mathbf{Pf}

. Then

f

is called a harmonic function for

\mathbf{P}

. (a) Show that for

f

harmonic

\mathbf{f} = \mathbf{P}^n \mathbf{f}

for all

n

. (b) Show, using (a), that for

f

harmonic

\mathbf{f} = \mathbf{P}^\infty \mathbf{f},

where

\mathbf{P}^\infty = \lim_{n \to \infty} \mathbf{P}^n = \begin{pmatrix} \mathbf{0} & \mathbf{B} \\ \mathbf{0} & \mathbf{I} \end{pmatrix}.

i

your expected final fortune

\sum_k b_{ik} f(k)

is equal to your starting fortune

f(i)

. In other words, a fair game on a finite state space remains fair to the end.