Linear Algebra and Its Applications · Problem 7.4.19

Linear Programming and Game Theory — Gilbert Strang

Mathematics · conjugate-gradient-orthogonality

Problem

In conjugate gradients, show that

r_1

is orthogonal to

r_0

(orthogonal residuals), and

p^T A p_0 = 0

(search directions are

A

-orthogonal). The iteration solves

Ax = b

by minimizing the error

e^T A e

in the Krylov subspace. It is a fantastic algorithm.