Callen H.B - Thermodynamics And An Introduction To Thermostatistics · Problem 17.1-3

QUANTUM FLUIDS

Physics — Statistical Mechanics · entropy-disorder-maximum

Problem

Prove that the quantity −k∑jfjln⁡fj-k\sum_j f_j \ln f_j is maximum if all the fjf_j are equal by applying the mathematical inequality valid for any continuous convex function Φ(x)\Phi(x) Φ(1Ω∑k=1Ωak)≤1Ω∑k=1ΩΦ(ak)\Phi\left(\frac{1}{\Omega}\sum_{k=1}^{\Omega} a_k\right) \leq \frac{1}{\Omega}\sum_{k=1}^{\Omega} \Phi(a_k) Give a graphical interpretation of the inequality.
Intuitive explanation

You're maximizing entropy by recognizing that the sum ∑jfjln⁡fj\sum_j f_j \ln f_j is minimized when probabilities are uniform—this follows from Jensen's inequality for convex functions. The key insight is identifying Φ(x)=ln⁡x\Phi(x) = \ln x as concave (so −ln⁡x-\ln x is convex), then applying the inequality to show that any non-uniform distribution yields a larger penalty term, making entropy smaller. Graphically, the inequality says a function's average value at points lies below the average of the function values themselves—the curve bends upward, so spreading points out increases the sum.

Intuitive explanation

You're maximizing entropy by recognizing that the sum ∑jfjln⁡fj\sum_j f_j \ln f_j is minimized when probabilities are uniform—this follows from Jensen's inequality for convex functions. The key insight is identifying Φ(x)=ln⁡x\Phi(x) = \ln x as concave (so −ln⁡x-\ln x is convex), then applying the inequality to show that any non-uniform distribution yields a larger penalty term, making entropy smaller. Graphically, the inequality says a function's average value at points lies below the average of the function values themselves—the curve bends upward, so spreading points out increases the sum.

Related theory

Browse the problem library