Introduction to Probability · Problem 11.1.13

This problem is about iterating a Markov chain forward in time. The key insight is that $\mathbf{u}^{(n)} = \mathbf{u} \mathbf{P}^n$—you're applying the transition matrix $\mathbf{P}$ repeatedly to see how a probability distribution evolves after $n$ steps. Starting from different initial states (like "definitely in state 2" or "equally likely in any state"), you'll watch the system converge toward its long-run behavior.

This problem is about iterating a Markov chain forward in time. The key insight is that $\mathbf{u}^{(n)} = \mathbf{u} \mathbf{P}^n$—you're applying the transition matrix $\mathbf{P}$ repeatedly to see how a probability distribution evolves after $n$ steps. Starting from different initial states (like "definitely in state 2" or "equally likely in any state"), you'll watch the system converge toward its long-run behavior.