Noise in a communication channel flips bits at random with probability P. Argue that the entropy associated with this process is
S=−PlogP−(1−P)log(1−P).(15.29)
It turns out that the rate R at which we can pass information along this noisy channel is 1−S. (This is an application of Shannon's noisy channel coding theorem, and a nice proof of this theorem is given on page 548 of Nielsen and Chuang (2000).)