Symmetric Matrices and the Spectral Theorem

⚡ Key Takeaways

Symmetric matrices ( $A = A^T$ ) are the most well-behaved matrices in linear algebra: the Spectral Theorem guarantees they have only real eigenvalues and an orthonormal basis of eigenvectors, so every real symmetric matrix can be written $A = Q\Lambda Q^T$ with $Q$ orthogonal and $\Lambda$ diagonal. Applies wherever quadratic forms, covariance, or self-adjoint operators appear — statistics, mechanics, optimization, quantum theory. Proven in stages by Cauchy, Jacobi, and Hilbert (19th–early 20th century).

💡 The Core Insight

Symmetry of a matrix is equivalent to a geometric condition: the associated linear map is self-adjoint with respect to the standard inner product, $\langle Ax, y\rangle = \langle x, Ay\rangle$ . This single identity is the engine that forces eigenvalues to be real, eigenvectors of distinct eigenvalues to be orthogonal, and — crucially — guarantees a complete orthonormal eigenbasis. Diagonalization becomes a rotation of coordinates in which the map acts by independent stretches along perpendicular axes.

DefinitionSymmetric Matrix

A matrix $A \in \mathbb{R}^{n\times n}$ is symmetric if it equals its own transpose:

1

\quad A = A^T \iff a_{ij} = a_{ji} \quad \text{for all } i,j.

Derivation hint: the transpose reflects entries across the main diagonal; equality means each off-diagonal pair matches. Equivalently, $A$ is symmetric iff it is self-adjoint for the Euclidean inner product:

2

\quad \langle Ax, y\rangle = \langle x, Ay\rangle \quad \text{for all } x,y \in \mathbb{R}^n.

Derivation hint: expand $\langle Ax,y\rangle = (Ax)^T y = x^T A^T y$ and $\langle x, Ay\rangle = x^T A y$ ; equality for all $x,y$ forces $A^T = A$ .

DefinitionOrthogonal Diagonalization

$A$ is orthogonally diagonalizable if there exist an orthogonal matrix $Q$ ( $Q^T Q = I$ ) and a diagonal matrix $\Lambda = \mathrm{diag}(\lambda_1,\dots,\lambda_n)$ such that

3

\quad A = Q\Lambda Q^T = \sum_{k=1}^{n} \lambda_k\, q_k q_k^T,

Derivation hint: the columns $q_k$ of $Q$ are orthonormal eigenvectors; $q_k q_k^T$ projects onto the line spanned by $q_k$ , so $A$ is a weighted sum of orthogonal projections. where the columns $q_1,\dots,q_n$ of $Q$ are orthonormal eigenvectors of $A$ satisfying $A q_k = \lambda_k q_k$ .

TheoremSpectral Theorem for Real Symmetric Matrices

Let $A \in \mathbb{R}^{n\times n}$ with $A = A^T$ . Then: 1. All eigenvalues of $A$ are real. 2. Eigenvectors corresponding to distinct eigenvalues are orthogonal. 3. There exists an orthonormal basis of $\mathbb{R}^n$ consisting of eigenvectors of $A$ ; equivalently, $A = Q\Lambda Q^T$ with $Q$ orthogonal and $\Lambda$ real diagonal. Conversely, any matrix of the form $Q\Lambda Q^T$ with $Q$ orthogonal and $\Lambda$ real diagonal is symmetric.

ProofReality of Eigenvalues

Let $\lambda \in \mathbb{C}$ be an eigenvalue with (possibly complex) eigenvector $x \neq 0$ , so $Ax = \lambda x$ .

Step 1. Take the conjugate-transpose inner product with $x$ :

4

\quad x^* A x = \lambda\, x^* x = \lambda \|x\|^2.

Derivation hint: multiply $Ax = \lambda x$ on the left by $x^* = \bar{x}^T$ .

Step 2. Take the conjugate transpose of the scalar $x^*Ax$ . Since it is a $1\times 1$ matrix it equals its own transpose, and because $A$ is real symmetric ( $A^* = \bar{A}^T = A^T = A$ ):

5

\quad (x^*Ax)^* = x^* A^* x = x^* A x.

Derivation hint: $(x^*Ax)^* = x^* A^* (x^*)^* = x^* A^* x$ ; use $A^*=A$ for real symmetric matrices.

Step 3. Thus $x^*Ax$ is real. From [4], $\lambda \|x\|^2$ is real and $\|x\|^2 > 0$ , so $\lambda \in \mathbb{R}$ . $\blacksquare$

∎

ProofOrthogonality of Distinct-Eigenvalue Eigenvectors

Let $Ax = \lambda x$ , $Ay = \mu y$ with $\lambda \neq \mu$ (both real).

Step 1. Using self-adjointness [2]:

6

\quad \lambda\langle x,y\rangle = \langle Ax, y\rangle = \langle x, Ay\rangle = \mu\langle x,y\rangle.

Derivation hint: substitute $Ax=\lambda x$ on the left and $Ay = \mu y$ on the right, pulling scalars out of the inner product.

Step 2. Therefore $(\lambda - \mu)\langle x,y\rangle = 0$ . Since $\lambda \neq \mu$ , we get $\langle x,y\rangle = 0$ . $\blacksquare$

∎

ProofExistence of an Orthonormal Eigenbasis — Induction

We prove the full diagonalization by induction on $n$ .

Base case $n=1$ : a $1\times 1$ matrix is trivially diagonal.

Inductive step. Assume the result for $(n-1)$ . By the fundamental theorem of algebra and reality (proven above), $A$ has a real eigenvalue $\lambda_1$ with a unit eigenvector $q_1$ , $Aq_1 = \lambda_1 q_1$ .

Step 1. Let $W = q_1^\perp = \{v : \langle v, q_1\rangle = 0\}$ , an $(n-1)$ -dimensional subspace. Claim: $W$ is invariant under $A$ . For $v \in W$ ,

7

\quad \langle Av, q_1\rangle = \langle v, A q_1\rangle = \lambda_1\langle v, q_1\rangle = 0,

Derivation hint: move $A$ across the inner product by self-adjointness, then use $Aq_1 = \lambda_1 q_1$ and $v \perp q_1$ . so $Av \in W$ .

Step 2. The restriction $A|_W : W \to W$ is a self-adjoint operator on an $(n-1)$ -dimensional inner product space. Choosing any orthonormal basis of $W$ , its matrix is symmetric, so by the inductive hypothesis $W$ has an orthonormal basis $q_2,\dots,q_n$ of eigenvectors of $A$ .

Step 3. Then $q_1, q_2,\dots,q_n$ is an orthonormal eigenbasis of $\mathbb{R}^n$ . Assembling them as columns of $Q$ gives $AQ = Q\Lambda$ , i.e. $A = Q\Lambda Q^T$ . $\blacksquare$

∎

mathematician · David Hilbert | 1862–1943

While Cauchy (1829) first proved the reality of eigenvalues of symmetric matrices in his study of quadratic surfaces, it was David Hilbert who, in his 1904–1910 work on integral equations, generalized the spectral theorem to infinite-dimensional self-adjoint operators, coining the term Spektrum. Hilbert's framework — later abstracted into Hilbert spaces by von Neumann — revealed the finite-dimensional Spectral Theorem as the simplest instance of a vast theory underlying quantum mechanics and functional analysis.

🧠 Intuition

Picture a symmetric matrix as describing a stretching of space along perpendicular axes. A general matrix can rotate and stretch in a tangled way (its singular vectors and eigenvectors differ), but a symmetric matrix decouples into clean orthogonal directions. The eigenvectors $q_k$ are the principal axes, and the eigenvalues $\lambda_k$ are the stretch factors. The decomposition $A = \sum_k \lambda_k q_k q_k^T$ literally says: project onto each principal axis, scale by $\lambda_k$ , and add up. The quadratic form $x^T A x = \sum_k \lambda_k\langle q_k, x\rangle^2$ becomes a sum of independent squares once you rotate into the eigenbasis — this is why symmetric matrices are exactly the ones whose level sets are ellipsoids (or hyperboloids), with axes along the eigenvectors.

Figure

Orthogonal eigenbasis of a symmetric matrix: the unit circle maps to an ellipse whose axes lie along the eigenvectors

q_1, q_2

, stretched by

\lambda_1, \lambda_2

.

ExampleDiagonalizing a 2×2 Symmetric Matrix

Given: $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ .

Step 1 — Characteristic polynomial.

8

\quad \det(A - \lambda I) = (2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = (\lambda-3)(\lambda-1).

Derivation hint: expand the determinant of $\begin{pmatrix}2-\lambda & 1\\1 & 2-\lambda\end{pmatrix}$ . So $\lambda_1 = 3$ , $\lambda_2 = 1$ (both real, as guaranteed).

Step 2 — Eigenvectors. For $\lambda_1 = 3$ : $(A-3I)v = \begin{pmatrix}-1 & 1\\1 & -1\end{pmatrix}v = 0 \Rightarrow v_1 = \tfrac{1}{\sqrt2}(1,1)^T$ . For $\lambda_2 = 1$ : $(A-I)v = \begin{pmatrix}1 & 1\\1 & 1\end{pmatrix}v = 0 \Rightarrow v_2 = \tfrac{1}{\sqrt2}(1,-1)^T$ . Note $\langle v_1, v_2\rangle = \tfrac12(1\cdot1 + 1\cdot(-1)) = 0$ — orthogonal, as guaranteed.

Step 3 — Assemble.

9

\quad Q = \frac{1}{\sqrt2}\begin{pmatrix}1 & 1\\1 & -1\end{pmatrix},\qquad \Lambda = \begin{pmatrix}3 & 0\\0 & 1\end{pmatrix}, \qquad A = Q\Lambda Q^T.

Derivation hint: columns of $Q$ are the normalized eigenvectors; verify $Q\Lambda Q^T$ reproduces $A$ by direct multiplication.

\boxed{A = \tfrac{1}{\sqrt2}\begin{pmatrix}1 & 1\\1 & -1\end{pmatrix}\begin{pmatrix}3 & 0\\0 & 1\end{pmatrix}\tfrac{1}{\sqrt2}\begin{pmatrix}1 & 1\\1 & -1\end{pmatrix}}

ExampleQuadratic Form and Principal Axes

Given: Classify the conic $5x^2 + 4xy + 5y^2 = 36$ .

Step 1 — Matrix form. Write $q(x,y) = \mathbf{x}^T A \mathbf{x}$ with the symmetric matrix

10

\quad A = \begin{pmatrix} 5 & 2 \\ 2 & 5 \end{pmatrix},

Derivation hint: the off-diagonal entry is half the coefficient of the cross-term $xy$ , split symmetrically.

Step 2 — Eigenvalues. $\det(A-\lambda I) = (5-\lambda)^2 - 4 = \lambda^2 - 10\lambda + 21 = (\lambda-7)(\lambda-3)$ , so $\lambda_1 = 7$ , $\lambda_2 = 3$ .

Step 3 — Rotate to principal axes. Eigenvectors $\tfrac1{\sqrt2}(1,1)^T$ (for $7$ ) and $\tfrac1{\sqrt2}(1,-1)^T$ (for $3$ ). In rotated coordinates $(u,v)$ :

11

\quad 7u^2 + 3v^2 = 36 \;\Longrightarrow\; \frac{u^2}{36/7} + \frac{v^2}{12} = 1.

Derivation hint: substitute $\mathbf{x} = Q\mathbf{u}$ ; the quadratic form becomes $\mathbf{u}^T\Lambda\mathbf{u} = \sum\lambda_k u_k^2$ . This is an ellipse with semi-axes $\sqrt{36/7}\approx 2.27$ and $\sqrt{12}\approx 3.46$ , rotated $45^\circ$ .

\boxed{\text{Ellipse: semi-axes } \sqrt{36/7} \text{ and } \sqrt{12}, \text{ axes along } (1,\pm1).}

⚠️ Common mistake

Symmetry depends on the inner product, not just the entries. Over $\mathbb{C}$ , the natural analogue of "symmetric" is Hermitian ( $A = A^*$ , i.e. $\overline{a_{ij}} = a_{ji}$ ), not $A = A^T$ . A complex matrix with $A = A^T$ need NOT have real eigenvalues or an orthonormal eigenbasis. The Spectral Theorem as stated requires real symmetric (or complex Hermitian); do not apply it to a complex symmetric matrix.

⚠️ Common mistake

Repeated eigenvalues do not break the theorem. Even when an eigenvalue has multiplicity $> 1$ , the eigenspace has full dimension equal to the multiplicity (no defective directions), so an orthonormal eigenbasis still exists. Within a repeated eigenspace you must apply Gram–Schmidt to manufacture orthonormal eigenvectors, since eigenvectors of the same eigenvalue need not be orthogonal automatically.

Counterexample

The matrix $N = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ is not symmetric. It has the single eigenvalue $0$ with a one-dimensional eigenspace spanned by $(1,0)^T$ — it is defective and cannot be diagonalized at all. This shows symmetry is essential: dropping it destroys both diagonalizability and the orthonormal eigenbasis. Likewise $\begin{pmatrix} 0 & -1 \\ 1 & 0\end{pmatrix}$ (antisymmetric) has eigenvalues $\pm i$ — non-real, confirming symmetry is what forces real spectra.

connection

The Spectral Theorem is the finite-dimensional heart of a web of results. It underlies the Singular Value Decomposition (apply spectral decomposition to $A^TA$ and $AA^T$ ). It generalizes to Hermitian Matrices over $\mathbb{C}$ and, infinitely, to the Spectral Theorem for Self-Adjoint Operators on Hilbert spaces. The connection between sign of eigenvalues and definiteness gives Positive Definite Matrices and the Quadratic Forms classification (Sylvester's law of inertia). The diagonalizing rotation is exactly the construction behind Principal Component Analysis, and the variational characterization of eigenvalues feeds the Rayleigh Quotient and Courant–Fischer Theorem.

🔬 Where this appears

⏳ History

- 1801–1822 — Gauss & Cauchy (quadratic forms): study of quadratic surfaces and their principal axes motivates the diagonalization of symmetric coefficient arrays.

- 1829 — Augustin-Louis Cauchy: proves that the eigenvalues of a real symmetric matrix (arising from quadric surfaces) are always real — the first version of the reality result.

- 1846 — Carl Jacobi: introduces iterative rotations (the Jacobi method) to numerically diagonalize symmetric matrices, a technique still used today.

- 1852 — James Joseph Sylvester: proves the law of inertia, showing the signs of eigenvalues (signature) are invariant under congruence.

- 1858 — Arthur Cayley: founds matrix algebra proper, giving the modern language of the transpose and matrix multiplication.

- 1870s — Karl Weierstrass & Leopold Kronecker: rigorize the theory of elementary divisors, clarifying when matrices are diagonalizable.

- 1904–1910 — David Hilbert: extends spectral ideas to infinite-dimensional integral operators, coining Spektrum and laying the groundwork for functional analysis.

- 1929–1932 — John von Neumann: establishes the abstract Spectral Theorem for unbounded self-adjoint operators on Hilbert space, providing the rigorous foundation of quantum mechanics.

open-problem

While the finite-dimensional Spectral Theorem is complete and elementary, its infinite-dimensional descendants remain subtle. For unbounded self-adjoint operators the "eigenvalues" may form a continuous spectrum with no eigenvectors in the Hilbert space, requiring the measure-theoretic spectral resolution $A = \int \lambda\, dE_\lambda$ . Determining the spectrum of specific operators (e.g. random Schrödinger operators, the question of Anderson localization, or the conjectured spectral interpretation of the Riemann zeta zeros via the Hilbert–Pólya conjecture) is an active research frontier. Numerically, computing eigendecompositions of very large symmetric matrices stably and at scale (e.g. for $10^9 \times 10^9$ sparse Laplacians) remains a driving challenge in scientific computing.

Symmetric Matrices and the Spectral Theorem

Formal Statement

Why It Holds

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also