Introduction to Probability · Problem 11.2.34

Markov Chains — Grinstead & Snell

Mathematics — Probability · absorbing-markov-chains

Problem

Complete the following alternate proof of Theorem 11.6. Let

s_i

be a transient state and

s_j

be an absorbing state. If we compute

b_{ij}

in terms of the possibilities on the outcome of the first step, then we have the equation

b_{ij} = p_{ij} + \sum_k p_{ik} b_{kj},

where the summation is carried out over all transient states

s_k

. Write this in matrix form, and derive from this equation the statement

\mathbf{B} = \mathbf{NR}