Linear algebra & the spectral theorem

⚡ Key Takeaways

One-line statement: Every self-adjoint (Hermitian) operator on a finite-dimensional inner product space has an orthonormal basis of eigenvectors, and is therefore diagonalizable by a unitary change of coordinates with real eigenvalues. Key result in words: Symmetric/Hermitian and, more generally, normal matrices are exactly those that can be diagonalized in an orthonormal basis; the geometry of the operator decomposes into independent stretching along mutually perpendicular axes. Where it applies: Quantum mechanics (observables), principal component analysis, vibration analysis, quadratic forms, Markov chains, differential equations, and any setting reducible to symmetric/Hermitian operators. Who/when: Roots in Cauchy (1829), Hermite, Jacobi; rigorous infinite-dimensional version by Hilbert, von Neumann (1929–1932).

💡 The Core Insight

A general linear map shears, rotates, and stretches space in entangled ways. But if the operator respects the inner product in the right way — if it is normal — then space splits into mutually orthogonal one-dimensional (or invariant) pieces on each of which the operator acts as pure scalar multiplication. The whole operator is then nothing but a weighted sum of orthogonal projections. Diagonalization with an orthonormal basis is geometry, not just bookkeeping: the eigenvectors are genuinely perpendicular directions of pure scaling.

DefinitionInner product space, adjoint, normal, self-adjoint

Let $V$ be a finite-dimensional vector space over $\mathbb{F}\in\{\mathbb{R},\mathbb{C}\}$ with inner product $\langle\cdot,\cdot\rangle$ (conjugate-linear in the first or second slot per convention; here linear in the second, conjugate-linear in the first).

The adjoint $T^{*}$ of $T:V\to V$ is the unique operator satisfying $\langle Tx,y\rangle=\langle x,T^{*}y\rangle$ for all $x,y$ .
$T$ is self-adjoint (Hermitian) if $T=T^{*}$ ; in matrix terms $A=A^{*}$ (conjugate transpose), and over $\mathbb{R}$ , $A=A^{T}$ (symmetric).
$T$ is unitary if $T^{*}T=TT^{*}=I$ (orthogonal over $\mathbb{R}$ ).
$T$ is normal if $T^{*}T=TT^{*}$ .
$T$ is skew-adjoint if $T^{*}=-T$ .

Self-adjoint, unitary, and skew-adjoint operators are all normal.

TheoremSpectral Theorem — Complex (Normal) Version

Let $V$ be a finite-dimensional complex inner product space and $T:V\to V$ linear. Then $T$ is normal ( $TT^{*}=T^{*}T$ ) if and only if there is an orthonormal basis $\{u_1,\dots,u_n\}$ of $V$ consisting of eigenvectors of $T$ . Equivalently, there exists a unitary $U$ and diagonal $\Lambda$ with

3

\quad A=U\Lambda U^{*}=\sum_{k=1}^{n}\lambda_{k}\,u_{k}u_{k}^{*}.

Derivation hint: $U=[u_1\,\cdots\,u_n]$ has orthonormal columns so $U^{*}U=I$ ; $AU=U\Lambda$ encodes $Au_k=\lambda_k u_k$ ; right-multiply by $U^{*}$ .

TheoremSpectral Theorem — Real Symmetric Version

Let $A$ be a real $n\times n$ symmetric matrix ( $A=A^{T}$ ). Then all eigenvalues of $A$ are real, and there exists an orthogonal matrix $Q$ ( $Q^{T}Q=I$ ) such that

4

\quad Q^{T}AQ=\Lambda=\operatorname{diag}(\lambda_1,\dots,\lambda_n),\qquad \lambda_k\in\mathbb{R}.

Derivation hint: Symmetric real matrices are Hermitian when viewed over $\mathbb{C}$ ; reality of eigenvalues plus the ability to choose real eigenvectors forces a real orthogonal diagonalizer.

TheoremSpectral Decomposition / Resolution of Identity

If $T$ is normal with distinct eigenvalues $\mu_1,\dots,\mu_m$ and $P_j$ is the orthogonal projection onto the eigenspace $E_{\mu_j}=\ker(T-\mu_j I)$ , then

5

\quad T=\sum_{j=1}^{m}\mu_j P_j,\qquad \sum_{j=1}^{m}P_j=I,\qquad P_jP_k=\delta_{jk}P_j,\qquad P_j^{*}=P_j.

Derivation hint: Group the rank-one terms $\lambda_k u_k u_k^{*}$ in [3] by equal eigenvalue; each group sums to an orthogonal projection onto an eigenspace.

ProofSpectral Theorem via Schur + Normality

We prove the complex normal version in numbered steps.

Step 1 (Schur triangularization). Over $\mathbb{C}$ every operator has at least one eigenvalue (fundamental theorem of algebra applied to $\det(A-\lambda I)=0$ ). By induction on $\dim V$ , choosing a unit eigenvector and restricting $T$ to the orthogonal complement of its span (after orthogonal projection), one builds an orthonormal basis in which $T$ is upper triangular:

6

\quad U^{*}AU=R,\qquad R\ \text{upper triangular},\ U\ \text{unitary}.

Derivation hint: Extend a unit eigenvector to an orthonormal basis; the first column of $U^*AU$ is $\lambda e_1$ ; recurse on the trailing $(n-1)\times(n-1)$ block.

Step 2 (Normality is preserved). Since $U$ is unitary, $R=U^{*}AU$ satisfies $R^{*}R=U^{*}A^{*}A\,U$ and $RR^{*}=U^{*}AA^{*}U$ . Because $A$ is normal, $A^{*}A=AA^{*}$ , hence $R^{*}R=RR^{*}$ : $R$ is normal.

Step 3 (Normal + triangular ⇒ diagonal). Compare diagonal entries of $R^{*}R$ and $RR^{*}$ . For the $(1,1)$ entry,

7

\quad (RR^{*})_{11}=\sum_{k}|r_{1k}|^{2},\qquad (R^{*}R)_{11}=|r_{11}|^{2}.

Derivation hint: $R$ upper triangular means row 1 may have many entries but column 1 has only $r_{11}$ ; write out the matrix products entrywise. Equality forces $r_{1k}=0$ for $k>1$ . Proceeding down the diagonal inductively kills every off-diagonal entry, so $R$ is diagonal. Thus $A=U\Lambda U^{*}$ , giving an orthonormal eigenbasis — the columns of $U$ .

Step 4 (Converse). If $A=U\Lambda U^{*}$ , then $A^{*}=U\Lambda^{*}U^{*}$ and since diagonal matrices commute, $A^{*}A=U\Lambda^{*}\Lambda U^{*}=U\Lambda\Lambda^{*}U^{*}=AA^{*}$ , so $A$ is normal. $\qquad\blacksquare$

∎

ProofEigenvalues of Self-Adjoint Operators are Real; Eigenvectors Orthogonal

Let $T=T^{*}$ , $Tx=\lambda x$ with $x\neq 0$ .

8

\quad \lambda\langle x,x\rangle=\langle Tx,x\rangle=\langle x,T^{*}x\rangle=\langle x,Tx\rangle=\overline{\lambda}\langle x,x\rangle.

Derivation hint: Use [1] with $y=x$ and self-adjointness; since $\langle x,x\rangle>0$ , conclude $\lambda=\overline\lambda$ . Hence $\lambda=\overline\lambda\in\mathbb{R}$ . For distinct eigenvalues $\lambda\neq\mu$ with $Tx=\lambda x$ , $Ty=\mu y$ :

9

\quad \lambda\langle y,x\rangle=\langle y,Tx\rangle=\langle Ty,x\rangle=\mu\langle y,x\rangle\ \Rightarrow\ (\lambda-\mu)\langle y,x\rangle=0\ \Rightarrow\ \langle y,x\rangle=0.

Derivation hint: Move $T$ across the inner product using self-adjointness and reality of $\mu$ ; nonzero $\lambda-\mu$ forces orthogonality. $\blacksquare$

∎

JV

John von Neumann

1903–1957

Von Neumann gave quantum mechanics its rigorous mathematical home in Mathematische Grundlagen der Quantenmechanik (1932), generalizing the finite-dimensional spectral theorem to unbounded self-adjoint operators on infinite-dimensional Hilbert space via the spectral measure $E(\cdot)$ , writing $T=\int_{\mathbb{R}}\lambda\,dE(\lambda)$ . In his framework physical observables are self-adjoint operators, their possible measured values are the spectrum, and the projection-valued measure encodes the probabilities of outcomes — the infinite-dimensional shadow of equation [5]. He also unified the Heisenberg matrix mechanics and Schrödinger wave mechanics by showing both act on the same Hilbert space.

🧠 Intuition

Think of a symmetric matrix as the quadratic form $q(x)=x^{T}Ax$ , e.g. the equation of an ellipse or ellipsoid. The level set $q(x)=1$ is a tilted ellipse; the spectral theorem says you can rotate your head (an orthogonal change of coordinates) until the ellipse's axes line up with the coordinate axes. In the new coordinates $q=\sum_k\lambda_k y_k^{2}$ — pure squares, no cross terms. The eigenvectors are the principal axes and the eigenvalues are $1/(\text{semi-axis})^{2}$ . Positive eigenvalues ⇒ ellipsoid, mixed signs ⇒ saddle (hyperboloid), a zero eigenvalue ⇒ a flat (degenerate) direction. Because $A$ is symmetric the axes are guaranteed perpendicular; a non-symmetric matrix can have eigenvectors at oblique angles, and you'd need a shear, not a rotation, to diagonalize it.

Figure

A real symmetric operator rotates to principal axes; the ellipse

x^TAx=1

has perpendicular eigen-axes scaled by

1/\sqrt{\lambda_i}

.

```svg

```

ExampleDiagonalizing a 2×2 real symmetric matrix

Given: $A=\begin{pmatrix}2&1\\1&2\end{pmatrix}=A^{T}$ .

Step 1 — Characteristic polynomial.

10

\quad \det(A-\lambda I)=(2-\lambda)^2-1=\lambda^2-4\lambda+3=(\lambda-1)(\lambda-3).

Derivation hint: Expand the determinant of $\begin{pmatrix}2-\lambda&1\\1&2-\lambda\end{pmatrix}$ . So $\lambda_1=3,\ \lambda_2=1$ (both real, as guaranteed).

Step 2 — Eigenvectors. For $\lambda_1=3$ : $(A-3I)v=0\Rightarrow\begin{pmatrix}-1&1\\1&-1\end{pmatrix}v=0\Rightarrow v_1=(1,1)$ . For $\lambda_2=1$ : $v_2=(1,-1)$ . These are orthogonal: $v_1\cdot v_2=0$ . ✓

Step 3 — Normalize and form $Q$ .

11

\quad Q=\frac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix},\qquad Q^{T}AQ=\begin{pmatrix}3&0\\0&1\end{pmatrix}.

Derivation hint: Divide each eigenvector by its norm $\sqrt2$ ; verify $Q^TAQ$ by direct multiplication.

Boxed answer: $\boxed{A=Q\Lambda Q^{T},\ \Lambda=\operatorname{diag}(3,1),\ Q=\tfrac{1}{\sqrt2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}.}$ The quadratic form becomes $3y_1^{2}+y_2^{2}$ .

ExampleSpectral projections and a function of a matrix

Given: Same $A$ as above. Compute $\sqrt{A}$ (positive square root) and verify the projection decomposition.

Step 1 — Projections. With $u_1=\tfrac1{\sqrt2}(1,1)^T$ , $u_2=\tfrac1{\sqrt2}(1,-1)^T$ :

12

\quad P_1=u_1u_1^{T}=\tfrac12\begin{pmatrix}1&1\\1&1\end{pmatrix},\qquad P_2=u_2u_2^{T}=\tfrac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix}.

Derivation hint: Outer product of each unit eigenvector with itself; check $P_1+P_2=I$ and $P_1P_2=0$ .

Step 2 — Spectral form. $A=3P_1+1\cdot P_2$ . Any function applies to eigenvalues:

13

\quad f(A)=f(3)P_1+f(1)P_2.

Derivation hint: $f(A)=Qf(\Lambda)Q^T$ ; grouping by eigenvalue reproduces the projection sum [5].

Step 3 — Square root. $f(\lambda)=\sqrt\lambda$ :

14

\quad \sqrt{A}=\sqrt3\,P_1+1\cdot P_2=\tfrac12\begin{pmatrix}\sqrt3+1&\sqrt3-1\\\sqrt3-1&\sqrt3+1\end{pmatrix}.

Derivation hint: Plug $f(3)=\sqrt3,\ f(1)=1$ into [13] using the matrices [12].

Boxed answer: $\boxed{\sqrt{A}\approx\begin{pmatrix}1.366&0.366\\0.366&1.366\end{pmatrix}},$ and one checks $(\sqrt A)^2=A$ exactly. (Units: dimensionless, as a pure matrix.)

⚠️ Common mistake

"Every diagonalizable matrix has orthogonal eigenvectors." False. $A=\begin{pmatrix}1&1\\0&2\end{pmatrix}$ is diagonalizable (distinct eigenvalues $1,2$ ) but its eigenvectors $(1,0)$ and $(1,1)$ are not orthogonal. Orthonormal diagonalizability is the special privilege of normal matrices. Also beware: "Hermitian ⇒ positive eigenvalues" is wrong — Hermitian only guarantees real eigenvalues; positivity requires the stronger condition $x^{*}Ax>0$ (positive definite).

CounterexampleDefective non-normal matrix

The matrix $N=\begin{pmatrix}0&1\\0&0\end{pmatrix}$ satisfies $N^2=0$ but $N\neq0$ . Its only eigenvalue is $0$ with a single one-dimensional eigenspace — there is no basis of eigenvectors at all, let alone an orthonormal one. Check normality:

15

\quad N^{*}N=\begin{pmatrix}0&0\\0&1\end{pmatrix}\neq\begin{pmatrix}1&0\\0&0\end{pmatrix}=NN^{*}.

Derivation hint: Compute both products directly; their inequality confirms $N$ is not normal, consistent with its non-diagonalizability. This shows the spectral theorem genuinely fails outside the normal class: such defective matrices need the Jordan Normal Form instead.

connection

The spectral theorem is the symmetric heart of several broader results. The Singular Value Decomposition generalizes it to arbitrary (even rectangular) matrices via $A=U\Sigma V^{*}$ , obtained by spectrally decomposing the symmetric $A^{*}A$ and $AA^{*}$ . The Principal Axis Theorem for quadratic forms is exactly the real symmetric case. The Cayley–Hamilton Theorem and minimal polynomials explain when diagonalization (with any basis) is possible, while the Jordan Normal Form handles the defective remainder. Over function spaces it extends to compact self-adjoint operators (Hilbert Space theory) giving eigenfunction expansions, and to unbounded operators underpinning Quantum Mechanics. The notion of Positive Definite Matrices is exactly "self-adjoint with positive spectrum."

🔬 Where this appears

⏳ History

- 1829 — Cauchy proves that a real symmetric (quadratic-form) matrix has real eigenvalues and orthogonal principal axes, in the context of the rotation of rigid bodies and conic sections.

- 1846 — Jacobi develops iterative rotation methods to diagonalize symmetric matrices (the Jacobi eigenvalue algorithm), giving constructive force to the theorem.

- 1855 — Hermite introduces what are now called Hermitian forms and proves reality of their eigenvalues, extending Cauchy's result to complex coefficients.

- 1858 — Cayley lays out matrix algebra; the characteristic equation and Cayley–Hamilton theorem clarify eigenvalue structure.

- 1870s–1880s — Weierstrass & Frobenius systematize elementary divisors and canonical forms, sharply distinguishing diagonalizable from defective matrices.

- 1904–1910 — Hilbert develops the spectral theory of infinite systems and integral equations, coining "spectrum" and treating quadratic forms in infinitely many variables.

- 1929 — von Neumann proves the spectral theorem for unbounded self-adjoint operators via projection-valued measures.

- 1932 — von Neumann's Mathematical Foundations of Quantum Mechanics canonizes the operator/observable correspondence.

- 1965 — Golub & Kahan / Francis (QR algorithm) make numerical spectral decomposition fast and stable, the standard in scientific computing today.

open-problem

The clean finite-dimensional statement frays at the edges. (1) Infinite dimensions: a self-adjoint operator may have continuous spectrum with no eigenvectors at all (e.g. the position operator $x\hat{}$ on $L^2(\mathbb{R})$ ), so the sum [5] must become an integral against a projection-valued measure, and unbounded operators force delicate domain considerations. (2) Numerical sensitivity: eigenvectors of matrices with near-degenerate eigenvalues are ill-conditioned, and clustered spectra are computationally hard to separate; quantifying optimal algorithms for the symmetric eigenvalue problem under finite precision remains an active area. (3) Joint diagonalization: several non-commuting normal matrices generally have no common eigenbasis (the algebraic root of the Heisenberg uncertainty principle), and finding approximate joint diagonalizers (used in blind source separation) is an open optimization frontier. (4) Non-normal operators governing transient growth in fluid dynamics are not captured by eigenvalues alone — pseudospectra are needed — showing the spectral theorem's reach ends precisely at non-normality.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also