Grinstead-Snell-Probability · Problem 11.2.24

Markov Chains

Mathematics · absorbing-markov-chains

Problem

Show that

w_x

of Exercise 23 satisfies the following conditions: (a)

w_x = pw_{x+1} + qw_{x-1}

for

x = 1, 2, \ldots, T-1

. (b)

w_0 = 0

. (c)

w_T = 1

. Show that these conditions determine

w_x

. Show that, if

p = q = 1/2

, then

w_x = \frac{x}{T}

satisfies (a), (b), and (c) and hence is the solution. If

p \neq q

, show that

w_x = \frac{(q/p)^x - 1}{(q/p)^T - 1}

satisfies these conditions and hence gives the probability of the gambler winning.