Linear Algebra and Its Applications · Problem 6.4.4

Computations with Matrices — Gilbert Strang

Mathematics · minimum-principles-least-squares

Problem

(Review) Another quadratic that certainly has its minimum at

Ax = b

Q(x) = \frac{1}{2}\|Ax - b\|^2 = \frac{1}{2}x^TA^TAx - x^TA^Tb + \frac{1}{2}b^Tb.

Comparing

Q

with

P

, and ignoring the constant

\frac{1}{2}b^Tb

, what system of equations do we get at the minimum of

Q

? What are these equations called in the theory of least squares?