Eigenvalues, Eigenvectors, and Diagonalization

⚡ Key Takeaways

For a square matrix $A$ , an eigenvector $v \neq 0$ satisfies $Av = \lambda v$ for a scalar eigenvalue $\lambda$ — $A$ acts on $v$ by pure scaling. Diagonalization writes $A = PDP^{-1}$ when enough independent eigenvectors exist, turning matrix powers and exponentials into trivial scalar operations. Applies everywhere: differential equations, quantum mechanics, vibrations, PageRank, principal component analysis. Foundational result of linear algebra, crystallized in the 18th–20th centuries.

💡 The Core Insight

A matrix is a verb — it transforms vectors. Eigenvectors are the directions on which that verb degenerates into multiplication by a number. If you change to a basis built from eigenvectors, the tangled action of $A$ unravels into $n$ independent one-dimensional scalings. Diagonalization is a change of viewpoint: the same operator looks like a diagonal matrix when you stand on its natural axes.

DefinitionEigenvalue and Eigenvector

Let $A$ be an $n \times n$ matrix over a field $\mathbb{F}$ (typically $\mathbb{R}$ or $\mathbb{C}$ ). A scalar $\lambda \in \mathbb{F}$ is an eigenvalue of $A$ if there exists a nonzero vector $v \in \mathbb{F}^n$ , the eigenvector, such that

1

\quad Av = \lambda v, \qquad v \neq 0.

Derivation hint: this is the defining equation; the nonzero requirement excludes the trivial $A\cdot 0 = \lambda\cdot 0$ which holds for every $\lambda$ . Symbols: $A$ = the operator ( $n\times n$ ); $v$ = eigenvector (nonzero column); $\lambda$ = eigenvalue (scalar); $n$ = dimension; $\mathbb{F}$ = scalar field.

DefinitionCharacteristic Polynomial and Equation

Rearranging [1] gives $(A - \lambda I)v = 0$ . A nonzero solution exists iff $A - \lambda I$ is singular, i.e.

2

\quad p_A(\lambda) = \det(A - \lambda I) = 0.

Derivation hint: a homogeneous system $Mv=0$ has a nontrivial solution iff $\det M = 0$ ; set $M = A-\lambda I$ . $p_A(\lambda)$ is the characteristic polynomial, a degree- $n$ polynomial in $\lambda$ . Its roots are exactly the eigenvalues. The eigenspace for $\lambda$ is

3

\quad E_\lambda = \ker(A - \lambda I) = \{ v : Av = \lambda v \},

Derivation hint: collect all eigenvectors for one $\lambda$ together with $0$ ; this kernel is a subspace because it is a null space. a subspace of $\mathbb{F}^n$ whose nonzero members are the eigenvectors. The dimension $\dim E_\lambda$ is the geometric multiplicity; the multiplicity of $\lambda$ as a root of $p_A$ is the algebraic multiplicity.

TheoremDiagonalization Theorem

An $n\times n$ matrix $A$ is diagonalizable over $\mathbb{F}$ iff it has $n$ linearly independent eigenvectors. Equivalently, $A$ is diagonalizable iff for every eigenvalue the geometric multiplicity equals the algebraic multiplicity and $p_A$ splits into linear factors over $\mathbb{F}$ . When this holds,

4

\quad A = PDP^{-1}, \qquad D = \operatorname{diag}(\lambda_1,\dots,\lambda_n),

Derivation hint: place eigenvectors as columns of $P$ ; then $AP = PD$ column-by-column from $Av_i=\lambda_i v_i$ , and invert $P$ . where the columns of $P$ are the eigenvectors and the diagonal entries of $D$ are the corresponding eigenvalues in matching order.

TheoremSpectral Theorem (Real Symmetric Case)

If $A$ is real and symmetric ( $A^\top = A$ ), then all eigenvalues are real and there exists an orthonormal basis of eigenvectors. Hence

5

\quad A = Q D Q^\top, \qquad Q^\top Q = I,

Derivation hint: symmetry forces real eigenvalues and orthogonality of eigenvectors from distinct eigenvalues; Gram–Schmidt handles repeated ones. with $Q$ orthogonal and $D$ real diagonal. Such matrices are always diagonalizable.

ProofDiagonalization ⇔ basis of eigenvectors

Step 1. Suppose $A$ has independent eigenvectors $v_1,\dots,v_n$ with $Av_i = \lambda_i v_i$ . Form $P = [\,v_1\ \cdots\ v_n\,]$ . Then

6

\quad AP = [\,Av_1\ \cdots\ Av_n\,] = [\,\lambda_1 v_1\ \cdots\ \lambda_n v_n\,] = PD.

Derivation hint: matrix–matrix product acts column-by-column; substitute each eigen-relation.

Step 2. Since the $v_i$ are independent, $P$ is invertible, so right-multiply by $P^{-1}$ :

7

\quad A = PDP^{-1}.

Derivation hint: from $AP=PD$ , multiply both sides on the right by $P^{-1}$ .

Step 3 (converse). If $A = PDP^{-1}$ with $D$ diagonal, then $AP = PD$ ; reading column $i$ gives $Av_i = D_{ii}v_i$ , so each column of $P$ is an eigenvector, and invertibility of $P$ makes them independent. $\blacksquare$

∎

ProofEigenvectors for distinct eigenvalues are independent

Step 1. Suppose $v_1,\dots,v_k$ are eigenvectors with distinct eigenvalues $\lambda_1,\dots,\lambda_k$ , and assume a dependence $c_1 v_1 + \cdots + c_k v_k = 0$ with not all $c_i$ zero; take the shortest such relation.

Step 2. Apply $A$ and also multiply the relation by $\lambda_1$ :

8

\quad \sum_i c_i \lambda_i v_i = 0, \qquad \sum_i c_i \lambda_1 v_i = 0.

Derivation hint: $A$ scales each $v_i$ by $\lambda_i$ ; the second equation is the original times the scalar $\lambda_1$ .

Step 3. Subtract:

9

\quad \sum_{i=2}^{k} c_i(\lambda_i - \lambda_1) v_i = 0.

Derivation hint: the $i=1$ term cancels because $\lambda_1-\lambda_1=0$ . This is a shorter relation. By minimality each $c_i(\lambda_i-\lambda_1)=0$ ; distinctness gives $\lambda_i\neq\lambda_1$ , so $c_i=0$ for $i\geq 2$ , then $c_1 v_1=0$ forces $c_1=0$ . Contradiction. $\blacksquare$

∎

WH

Werner Heisenberg

1901–1976

In 1925 Heisenberg reformulated quantum mechanics using arrays of numbers obeying non-commutative multiplication — what Born and Jordan recognized as matrices. Observables became Hermitian matrices, and the measurable values of a quantity are precisely its eigenvalues; the state collapses onto the corresponding eigenvector. The spectral theorem thus became the mathematical backbone of physical measurement, making eigenvalues among the most physically consequential objects in all of science.

🧠 Intuition

Picture $A$ as a rubber-sheet deformation of the plane. Most arrows you draw get both rotated and stretched. But a few special arrows only stretch — they don't swing. Those are eigenvectors; the stretch factor is the eigenvalue. A negative eigenvalue flips the arrow; $|\lambda|>1$ stretches; $|\lambda|<1$ shrinks; $\lambda=0$ collapses the direction entirely (so $A$ is singular). In the eigenvector basis the deformation is just "stretch axis 1 by $\lambda_1$ , axis 2 by $\lambda_2$ " — independent one-dimensional scalings, no coupling. Diagonalization is literally the act of tilting your head until the deformation looks like simple axis-aligned stretching.

Figure

A 2×2 matrix scales its eigenvectors and bends every other vector

ExampleDiagonalizing a 2×2 matrix

Given: $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ .

Step 1 — characteristic polynomial.

10

\quad \det(A-\lambda I) = (4-\lambda)(3-\lambda) - (1)(2) = \lambda^2 - 7\lambda + 10.

Derivation hint: expand the $2\times2$ determinant $(a-\lambda)(d-\lambda)-bc$ .

Step 2 — eigenvalues. $\lambda^2-7\lambda+10=(\lambda-5)(\lambda-2)=0$ , so $\lambda_1=5,\ \lambda_2=2$ .

Step 3 — eigenvectors. For $\lambda=5$ : $(A-5I)=\begin{pmatrix}-1&1\\2&-2\end{pmatrix}$ , giving $-x+y=0\Rightarrow v_1=(1,1)$ . For $\lambda=2$ : $(A-2I)=\begin{pmatrix}2&1\\2&1\end{pmatrix}$ , giving $2x+y=0\Rightarrow v_2=(1,-2)$ .

Step 4 — assemble.

11

\quad P=\begin{pmatrix}1&1\\1&-2\end{pmatrix},\quad D=\begin{pmatrix}5&0\\0&2\end{pmatrix},\quad A=PDP^{-1}.

Derivation hint: columns of $P$ are eigenvectors in the same order as the diagonal of $D$ .

\boxed{\lambda_1=5,\ v_1=(1,1);\quad \lambda_2=2,\ v_2=(1,-2);\quad A=PDP^{-1}.}

ExampleMatrix power via diagonalization

Given: the same $A$ ; compute $A^4$ acting on $x_0=(3,0)$ .

Step 1 — power rule. Since $A=PDP^{-1}$ ,

12

\quad A^k = PD^kP^{-1}, \qquad D^k=\operatorname{diag}(\lambda_1^k,\lambda_2^k).

Derivation hint: $A^k=(PDP^{-1})^k=PD(P^{-1}P)D\cdots P^{-1}=PD^kP^{-1}$ ; inner $P^{-1}P$ telescope to $I$ .

Step 2 — coordinates. Write $x_0$ in the eigenbasis: solve $c_1 v_1 + c_2 v_2 = (3,0)$ . $c_1+c_2=3,\ c_1-2c_2=0\Rightarrow c_1=2,\ c_2=1$ .

Step 3 — scale and recombine.

13

\quad A^4 x_0 = c_1\lambda_1^4 v_1 + c_2\lambda_2^4 v_2 = 2\cdot 625\,(1,1) + 1\cdot 16\,(1,-2).

Derivation hint: each eigen-component is just multiplied by $\lambda^4$ — that is the whole point of diagonalization.

\boxed{A^4 x_0 = (1250+16,\ 1250-32) = (1266,\ 1218).}

⚠️ Common mistake

Distinct eigenvalues are sufficient but NOT necessary for diagonalizability. If an $n\times n$ matrix has $n$ distinct eigenvalues it is diagonalizable — but a matrix with repeated eigenvalues can still be diagonalizable (the identity matrix $I$ has $\lambda=1$ repeated $n$ times and is already diagonal). Repeated eigenvalues only threaten diagonalizability; they don't forbid it. Always check whether geometric multiplicity equals algebraic multiplicity.

⚠️ Common mistake

Eigenvectors are not unique, but eigenvalues are. Any nonzero scalar multiple of an eigenvector is again an eigenvector for the same $\lambda$ (the whole line — eigenspace — qualifies). So "the eigenvector" is loose language; normalize ( $\|v\|=1$ ) if you need a canonical choice. Also: a $0$ eigenvalue is perfectly legitimate (it means $A$ is singular), but the zero vector is never an eigenvector.

CounterexampleA non-diagonalizable matrix

The shear $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ has characteristic polynomial $(1-\lambda)^2$ , so $\lambda=1$ with algebraic multiplicity 2. But

14

\quad (A-I)=\begin{pmatrix}0&1\\0&0\end{pmatrix}, \quad \dim\ker(A-I)=1.

Derivation hint: the only solutions of $(A-I)v=0$ are multiples of $(1,0)$ — a single eigenvector direction. Geometric multiplicity 1 $<$ algebraic multiplicity 2, so there is no basis of eigenvectors and $A$ is not diagonalizable. Such "defective" matrices require the Jordan Canonical Form instead.

connection

Eigen-theory threads through the entire subject. The Characteristic Polynomial packages eigenvalues as its roots, and the Cayley–Hamilton Theorem says every matrix satisfies its own characteristic equation. The Determinant equals the product of eigenvalues and the Trace equals their sum — instant sanity checks. For non-diagonalizable matrices the Jordan Canonical Form is the nearest substitute, while the Singular Value Decomposition generalizes the spectral theorem to any (even rectangular) matrix. Diagonalization is the engine behind Systems of Linear Differential Equations and the Matrix Exponential.

🔬 Where this appears

⏳ History

- 1750s — Euler & d'Alembert: encounter "principal axes" of rotating rigid bodies and oscillating systems — eigenvalues in disguise as characteristic frequencies.

- 1760s — Lagrange: studies the secular equation for planetary orbits, an early characteristic equation.

- 1829 — Cauchy: proves real symmetric ("self-adjoint") systems have real eigenvalues; foundational to the spectral theorem.

- 1840s — Sylvester & Cayley: develop matrix algebra; Cayley states (1858) what becomes the Cayley–Hamilton theorem.

- 1878 — Frobenius: rigorous theory of minimal/characteristic polynomials and canonical forms.

- 1904 — Hilbert: coins Eigenwert ("own/characteristic value") while studying integral operators; the German prefix eigen- survives untranslated in English.

- 1907 — Schur: the Schur decomposition — every complex matrix is unitarily triangularizable — underpins numerical eigenvalue methods.

- 1925 — Heisenberg, Born, Jordan: matrix mechanics makes eigenvalues the literal observables of physics.

- 1961 — Francis & Kublanovskaya: independently invent the QR algorithm, the workhorse for computing eigenvalues numerically to this day.

open-problem

Practical eigenvalue computation is delicate. Eigenvalues can be ill-conditioned: tiny perturbations to a non-normal matrix can move them enormously (the field of pseudospectra studies this), so naively rooting the characteristic polynomial is numerically disastrous — never do it for large $n$ . The general degree- $\geq 5$ characteristic polynomial has no radical formula (Abel–Ruffini), so all robust eigenvalue methods are iterative (QR, Lanczos, Arnoldi). For very large sparse matrices, computing the full spectrum is infeasible and one settles for a few extremal eigenvalues; efficient, accurate algorithms for clustered or nearly-defective spectra remain an active research frontier in numerical linear algebra.

Eigenvalues, Eigenvectors, and Diagonalization

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also