Eigenvalues & Eigenvectors

⚡ Key Takeaways

One-line: An eigenvector of a square matrix $A$ is a nonzero vector whose direction is unchanged when multiplied by $A$ , and its eigenvalue is the scalar by which it is stretched: $A\mathbf{v}=\lambda\mathbf{v}$ .

Key result in words: For an $n\times n$ matrix, eigenvalues are the roots of the characteristic polynomial $\det(A-\lambda I)=0$ ; each eigenvalue has an associated subspace of eigenvectors. When enough independent eigenvectors exist, the matrix diagonalizes, turning matrix powers and exponentials into trivial scalar operations.

Where it applies: Differential equations, vibrational analysis, quantum mechanics, principal component analysis, Markov chains, Google PageRank, stability theory.

Who/when: Concept crystallized through Cauchy, Jacobi, and Hilbert in the 19th–early 20th century; the German eigen ("own/characteristic") was popularized by Hilbert (1904).

💡 The Core Insight

Multiplication by a matrix usually mixes directions together. Eigenvectors are the directions the matrix refuses to mix — it only rescales them. If you choose a basis made of eigenvectors, the matrix becomes diagonal in that basis. Everything hard about a matrix (its powers, its exponential, its long-term behavior) becomes easy once you see it through its eigenstructure.

DefinitionEigenvalue and Eigenvector

Let $A$ be an $n\times n$ matrix over a field $\mathbb{F}$ (typically $\mathbb{R}$ or $\mathbb{C}$ ). A scalar $\lambda\in\mathbb{F}$ is an eigenvalue of $A$ if there exists a nonzero vector $\mathbf{v}\in\mathbb{F}^n$ such that

1

\quad A\mathbf{v}=\lambda\mathbf{v},\qquad \mathbf{v}\neq\mathbf{0}.

Derivation hint: This is the defining relation — $A$ acts on $\mathbf{v}$ purely as scalar multiplication by $\lambda$ . The condition $\mathbf{v}\neq\mathbf{0}$ excludes the trivial solution that holds for every $\lambda$ . Such $\mathbf{v}$ is an eigenvector associated with $\lambda$ . Symbols: $A$ = the matrix (linear operator); $\mathbf{v}$ = eigenvector (direction); $\lambda$ = eigenvalue (scale factor); $I$ = $n\times n$ identity; $n$ = dimension. Rearranging [1] gives the homogeneous system

2

\quad (A-\lambda I)\mathbf{v}=\mathbf{0}.

Derivation hint: Subtract $\lambda\mathbf{v}=\lambda I\mathbf{v}$ from both sides of [1] and factor the common $\mathbf{v}$ on the right. A nonzero solution exists iff the coefficient matrix is singular:

3

\quad \det(A-\lambda I)=0.

Derivation hint: A homogeneous square system $M\mathbf{v}=\mathbf{0}$ has a nonzero solution exactly when $M$ is not invertible, i.e. $\det M=0$ . The left side, expanded, is the characteristic polynomial $p_A(\lambda)$ :

4

\quad p_A(\lambda)=\det(A-\lambda I)=(-1)^n\big(\lambda^n - (\operatorname{tr}A)\,\lambda^{n-1}+\cdots+(-1)^n\det A\big).

Derivation hint: Expanding the determinant of $A-\lambda I$ by cofactors yields a degree- $n$ polynomial in $\lambda$ ; the leading coefficient is $(-1)^n$ , the $\lambda^{n-1}$ coefficient is $\pm\operatorname{tr}A$ , and the constant term is $\det A$ up to sign.

DefinitionEigenspace, Multiplicities

For an eigenvalue $\lambda$ , the eigenspace is the null space

5

\quad E_\lambda=\ker(A-\lambda I)=\{\mathbf{v}: A\mathbf{v}=\lambda\mathbf{v}\}.

Derivation hint: The set of all eigenvectors of $\lambda$ , together with $\mathbf{0}$ , is precisely the solution set of [2], which is a subspace. The algebraic multiplicity $m_a(\lambda)$ is the multiplicity of $\lambda$ as a root of $p_A$ . The geometric multiplicity $m_g(\lambda)=\dim E_\lambda$ . They satisfy

6

\quad 1\le m_g(\lambda)\le m_a(\lambda).

Derivation hint: A root contributes at least one eigenvector ( $m_g\ge1$ ), but a generalized-eigenvector argument (or the Jordan form) shows the geometric count can never exceed the algebraic count.

TheoremDiagonalizability

An $n\times n$ matrix $A$ is diagonalizable (there exists invertible $P$ with $P^{-1}AP=D$ diagonal) iff $A$ has $n$ linearly independent eigenvectors, equivalently iff $m_g(\lambda)=m_a(\lambda)$ for every eigenvalue. In that case

7

\quad A=PDP^{-1},\qquad D=\operatorname{diag}(\lambda_1,\dots,\lambda_n),

where the columns of $P$ are the eigenvectors and the diagonal of $D$ holds the corresponding eigenvalues. Derivation hint: Stack the relations $A\mathbf{v}_i=\lambda_i\mathbf{v}_i$ into $AP=PD$ column-by-column; invertibility of $P$ (guaranteed by independence) gives $A=PDP^{-1}$ .

TheoremSpectral Theorem (real symmetric case)

If $A$ is real and symmetric ( $A=A^{\mathsf T}$ ), then all eigenvalues are real and $A$ admits an orthonormal eigenbasis. There is an orthogonal matrix $Q$ ( $Q^{\mathsf T}Q=I$ ) with

8

\quad A=QDQ^{\mathsf T},\qquad D \text{ real diagonal}.

Derivation hint: Reality of eigenvalues follows from $\lambda\|\mathbf v\|^2=\mathbf v^*A\mathbf v=\overline{\lambda}\|\mathbf v\|^2$ ; orthogonality of eigenvectors for distinct eigenvalues comes from $(\lambda_i-\lambda_j)\langle\mathbf v_i,\mathbf v_j\rangle=0$ .

ProofFrom the definition to the spectrum

Step 1 — Reduce to a null-space condition. Equation [1] $A\mathbf v=\lambda\mathbf v$ is equivalent to $(A-\lambda I)\mathbf v=\mathbf 0$ (equation [2]). We seek a nonzero $\mathbf v$ .

Step 2 — Singularity criterion. A square homogeneous system has a nontrivial solution iff its matrix is singular. Hence $\lambda$ is an eigenvalue iff $\det(A-\lambda I)=0$ (equation [3]).

Step 3 — The characteristic polynomial. Expanding $\det(A-\lambda I)$ gives a monic-up-to-sign degree- $n$ polynomial $p_A(\lambda)$ (equation [4]). By the Fundamental Theorem of Algebra, over $\mathbb{C}$ it has exactly $n$ roots counted with multiplicity, so every complex matrix has $n$ eigenvalues.

Step 4 — Eigenvectors per eigenvalue. For each root $\lambda$ , solve $(A-\lambda I)\mathbf v=\mathbf 0$ . The solution set $E_\lambda$ is a subspace of dimension $m_g(\lambda)\ge1$ (since $A-\lambda I$ is singular, its null space is nontrivial).

Step 5 — Independence across distinct eigenvalues. Suppose $\mathbf v_1,\dots,\mathbf v_k$ are eigenvectors for distinct eigenvalues $\lambda_1,\dots,\lambda_k$ . Assume a dependence $\sum c_i\mathbf v_i=\mathbf 0$ with not all $c_i=0$ ; choose a minimal such relation. Apply $A$ : $\sum c_i\lambda_i\mathbf v_i=\mathbf 0$ . Subtract $\lambda_1\times$ (original): $\sum_{i\ge2} c_i(\lambda_i-\lambda_1)\mathbf v_i=\mathbf 0$ , a shorter relation — contradiction. Hence eigenvectors from distinct eigenvalues are linearly independent.

Step 6 — Diagonalization. If the independent eigenvectors number $n$ , they form a basis; collecting them as columns of $P$ gives $AP=PD$ , so $A=PDP^{-1}$ (equation [7]). $\blacksquare$

∎

mathematician · David Hilbert | 1862–1943

While the algebraic ideas trace to Cauchy and the term "characteristic value" was in use, it was David Hilbert who, in his 1904 work on integral equations, systematically used the prefix eigen ("own," "proper," "characteristic") — Eigenwert for eigenvalue and Eigenfunktion for eigenfunction. His spectral theory of self-adjoint operators in infinite dimensions generalized finite-dimensional eigen-analysis and laid the mathematical groundwork later seized upon by quantum mechanics, where observables are self-adjoint operators and measured values are their eigenvalues.

🧠 Intuition

Picture $A$ as a machine that deforms space — stretching, shearing, rotating. Most arrows get tilted and rescaled when fed through. Eigenvectors are the rare arrows that come out pointing the same way (or exactly reversed), only longer or shorter. The stretch factor is $\lambda$ .

$\lambda>1$ : stretch along that direction.
$0<\lambda<1$ : compress.
$\lambda<0$ : flip and rescale.
$\lambda=0$ : collapse that direction to zero (so $A$ is singular).
complex $\lambda$ : rotation mixed with scaling (no real fixed direction).

If you tilt your head so your coordinate axes line up with the eigenvectors, the matrix suddenly looks diagonal — it just scales each new axis independently. That is the entire payoff of Diagonalization.

Figure

Eigenvectors of a shear-and-stretch matrix: most vectors rotate, but the eigen-directions only rescale.

ExampleA 2×2 matrix with distinct real eigenvalues

Given: $A=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ (symmetric). Find eigenvalues, eigenvectors, and diagonalize.

Step 1 — Characteristic polynomial.

\det(A-\lambda I)=\det\begin{pmatrix}2-\lambda&1\\1&2-\lambda\end{pmatrix}=(2-\lambda)^2-1=\lambda^2-4\lambda+3.

Step 2 — Roots. $\lambda^2-4\lambda+3=(\lambda-1)(\lambda-3)=0\Rightarrow \lambda_1=1,\ \lambda_2=3.$

Step 3 — Eigenvector for $\lambda_1=1$ . Solve $(A-I)\mathbf v=\mathbf 0$ : $\begin{pmatrix}1&1\\1&1\end{pmatrix}\mathbf v=\mathbf0\Rightarrow v_1+v_2=0$ , so $\mathbf v_1=\begin{pmatrix}1\\-1\end{pmatrix}$ .

Step 4 — Eigenvector for $\lambda_2=3$ . Solve $\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\mathbf v=\mathbf0\Rightarrow v_1=v_2$ , so $\mathbf v_2=\begin{pmatrix}1\\1\end{pmatrix}$ .

Step 5 — Diagonalize. $P=\begin{pmatrix}1&1\\-1&1\end{pmatrix}$ , $D=\begin{pmatrix}1&0\\0&3\end{pmatrix}$ . Note $\mathbf v_1\cdot\mathbf v_2=0$ (orthogonal, as the spectral theorem guarantees).

\boxed{\lambda_1=1,\ \mathbf v_1=(1,-1)^{\mathsf T};\quad \lambda_2=3,\ \mathbf v_2=(1,1)^{\mathsf T};\quad A=PDP^{-1}.}

Check: $\operatorname{tr}A=4=1+3$ ✓, $\det A=3=1\cdot3$ ✓ (dimensionless scalars).

ExampleA defective matrix and a matrix power

Given: $A=\begin{pmatrix}3&1\\0&3\end{pmatrix}$ . Find eigenstructure and decide diagonalizability; then compute $A^{4}$ for a diagonalizable sibling.

Step 1 — Eigenvalues. $\det(A-\lambda I)=(3-\lambda)^2=0\Rightarrow\lambda=3$ with $m_a=2$ .

Step 2 — Eigenspace. $(A-3I)=\begin{pmatrix}0&1\\0&0\end{pmatrix}$ , null space spanned by $(1,0)^{\mathsf T}$ only, so $m_g=1$ .

Step 3 — Conclusion. Since $m_g=1<2=m_a$ , $A$ is not diagonalizable (it is defective; it needs a Jordan Canonical Form block).

Step 4 — Powers via diagonalization (sibling). For the diagonalizable $B=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ from the previous example, $B^4=PD^4P^{-1}$ with $D^4=\operatorname{diag}(1^4,3^4)=\operatorname{diag}(1,81)$ . Using $P^{-1}=\tfrac12\begin{pmatrix}1&-1\\1&1\end{pmatrix}$ :

B^4=\frac12\begin{pmatrix}1&1\\-1&1\end{pmatrix}\begin{pmatrix}1&0\\0&81\end{pmatrix}\begin{pmatrix}1&-1\\1&1\end{pmatrix}=\frac12\begin{pmatrix}82&80\\80&82\end{pmatrix}=\begin{pmatrix}41&40\\40&41\end{pmatrix}.

\boxed{A \text{ defective } (m_g=1); \quad B^4=\begin{pmatrix}41&40\\40&41\end{pmatrix}.}

⚠️ Common mistake

" $\mathbf v=\mathbf 0$ is an eigenvector." Never. The zero vector satisfies $A\mathbf 0=\lambda\mathbf 0$ for all $\lambda$ , which would make the concept meaningless. Eigenvectors are required nonzero. (But $\lambda=0$ is a legitimate eigenvalue — it just means $A$ is singular.)
"Eigenvectors are unique." Any nonzero scalar multiple of an eigenvector is also an eigenvector for the same $\lambda$ ; eigenvectors are defined only up to scale (and up to a basis choice within a multi-dimensional eigenspace).
"Every matrix is diagonalizable." False — defective matrices (geometric $<$ algebraic multiplicity) are not. See the previous example.
"Real matrices have real eigenvalues." Only guaranteed for symmetric (more generally normal Hermitian) matrices. A rotation matrix has complex eigenvalues.
"Algebraic multiplicity = geometric multiplicity always." Only when the matrix is diagonalizable; equation [6] allows strict inequality.

CounterexampleA real matrix with no real eigenvectors

The $90^\circ$ rotation $R=\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ rotates every vector, so no real vector keeps its direction — there is no real eigenvector. Yet its characteristic polynomial is $\lambda^2+1=0$ , giving complex eigenvalues $\lambda=\pm i$ with complex eigenvectors $(1,\mp i)^{\mathsf T}$ . This shows eigenvalues live naturally over $\mathbb{C}$ even when $A$ is real, and that real geometry can completely conceal them.

connection

Eigen-analysis is the hub of linear algebra. The Characteristic Polynomial supplies the eigenvalues; Diagonalization uses eigenvectors to simplify a matrix; the Spectral Theorem sharpens this for symmetric/Hermitian matrices into orthogonal diagonalization. When diagonalization fails, the Jordan Canonical Form gives the next-best structure. The Singular Value Decomposition generalizes eigenvalues to all (even non-square) matrices via eigenvalues of $A^{\mathsf T}A$ . In analysis, eigenvalues drive the solution of Systems of Linear Differential Equations through the Matrix Exponential, and Quadratic Forms are classified (positive-definite etc.) by the signs of eigenvalues. The Determinant equals the product and the Trace equals the sum of eigenvalues.

🔬 Where this appears

⏳ History

- 1760s — Euler & Lagrange: principal axes of rotating rigid bodies and of quadratic surfaces — implicit eigen-directions.

- 1820s — Cauchy: proves real symmetric matrices have real "characteristic values"; introduces the term valeurs caractéristiques and the characteristic equation.

- 1846 — Jacobi: the iterative method for diagonalizing symmetric matrices (Jacobi rotations), still used numerically.

- 1858 — Cayley: matrix algebra and the Cayley–Hamilton theorem (a matrix satisfies its own characteristic polynomial).

- 1870 — Jordan: the Jordan canonical form, handling non-diagonalizable (defective) matrices.

- 1904 — Hilbert: coins/popularizes Eigenwert and Eigenfunktion in the theory of integral equations; founds spectral theory.

- 1925–1932 — Heisenberg, Born, von Neumann: eigenvalues become physical observables; von Neumann's Mathematical Foundations of Quantum Mechanics (1932) puts spectral theory at the center of physics.

- 1929 — von Mises: the power iteration method for computing dominant eigenvalues.

- 1961 — Francis & Kublanovskaya: the QR algorithm, the workhorse of modern numerical eigenvalue computation.

open-problem

Conditioning and non-normality. Eigenvalues of non-normal matrices ( $AA^{\mathsf T}\neq A^{\mathsf T}A$ ) can be wildly sensitive to tiny perturbations, and eigenvectors can be nearly parallel, making numerical computation treacherous. This motivated pseudospectra (Trefethen): the set of eigenvalues reachable by small perturbations, which better predicts transient behavior than the spectrum alone.

Infinite dimensions. For operators on function spaces, the spectrum need not consist of eigenvalues — there is continuous and residual spectrum (e.g., the position operator in quantum mechanics has no eigenfunctions in $L^2$ ). Generalizing eigen-analysis to unbounded self-adjoint operators is the deep content of functional analysis.

Inverse eigenvalue problems. Reconstructing a matrix (or a physical structure) from prescribed eigenvalues is generally hard and not uniquely solvable — "Can one hear the shape of a drum?" (Kac, 1966) was answered no in 1992: distinct shapes can share the same Laplacian spectrum.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also