Introduction to Probability · Problem 11.1.7

Markov Chains — Grinstead & Snell

Mathematics — Probability · markov-chains-introduction

Problem

Find the matrices

\mathbf{P}^2

\mathbf{P}^3

\mathbf{P}^4

, and

\mathbf{P}^n

for the Markov chain determined by the transition matrix

\mathbf{P} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}

. Do the same for the transition matrix

\mathbf{P} = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}

. Interpret what happens in each of these processes.