Convex Optimization · Problem 9.6

Unconstrained minimization — Boyd & Vandenberghe

Mathematics — Optimization · gradient-descent-quadratic

Problem

Consider the quadratic objective function

f(x) = \frac{1}{2}(x_1^2 + \gamma x_2^2)

\mathbf{R}^2

, where

\gamma > 0

. Apply the gradient descent method with exact line search, starting at

x^{(0)} = (\gamma, 1)

. Derive the following closed-form expressions for the iterates

x^{(k)}

and their function values:

x_1^{(k)} = \gamma \left(\frac{\gamma-1}{\gamma+1}\right)^k, \quad x_2^{(k)} = \left(-\frac{\gamma-1}{\gamma+1}\right)^k,

and

f(x^{(k)}) = \frac{\gamma(\gamma+1)}{2}\left(\frac{\gamma-1}{\gamma+1}\right)^{2k} = \left(\frac{\gamma-1}{\gamma+1}\right)^{2k} f(x^{(0)}).