Grinstead-Snell-Probability · Problem 11.4.1

Markov Chains

Mathematics · fundamental-limit-theorem-computation

Problem

Define

\mathbf{P}

and

\mathbf{y}

\mathbf{P} = \begin{pmatrix} .5 & .5 \\ .25 & .75 \end{pmatrix}, \quad \mathbf{y} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}.

Compute

\mathbf{P}\mathbf{y}

\mathbf{P}^2\mathbf{y}

, and

\mathbf{P}^4\mathbf{y}

and show that the results are approaching a constant vector. What is this vector?