Introduction to Probability · Problem 11.17

Random Walks — Grinstead & Snell

Mathematics — Probability · markov-chains-mean-first-passage

Problem

An ergodic Markov chain is started in equilibrium (i.e., with initial probability vector

\mathbf{w}

). The mean time until the next occurrence of state

s_i

\bar{m}_i = \sum_k w_k m_{ki} + w_i r_i

. Show that

\bar{m}_i = z_{ii}/w_i

, by using the facts that

\mathbf{w}\mathbf{Z} = \mathbf{w}

and

m_{ki} = (z_{ii} - z_{ki})/w_i