Eigenvalues and Eigenvectors

⚡ Key Takeaways

One-line statement: For a square matrix $A$ , an eigenvector $\mathbf{v} \neq \mathbf{0}$ satisfies $A\mathbf{v} = \lambda\mathbf{v}$ , meaning $A$ merely scales $\mathbf{v}$ by the eigenvalue $\lambda$ without changing its direction. Key result in words: Eigenvalues are the roots of the characteristic polynomial $\det(A - \lambda I) = 0$ ; eigenvectors span the directions left invariant by the linear map. Where it applies: Diagonalization, differential equations, vibrations, quantum mechanics, PCA, Google's PageRank, stability analysis. Who/when: Roots in 18th-century work by Euler and Lagrange on rotation and quadratic forms; named ("eigen-" = "own/characteristic") by David Hilbert (1904).

💡 The Core Insight

Most vectors get knocked off their original line when you apply a matrix. Eigenvectors are the rare vectors that stay on their own line — the matrix only rescales them. If you can find enough of these invariant directions to build a basis, the matrix becomes diagonal in that basis, and applying it (or its powers, or its exponential) reduces to scalar arithmetic on each axis. Eigen-analysis is therefore the search for the coordinate system in which a linear map is as simple as possible.

DefinitionEigenvalue and Eigenvector

Let $A \in \mathbb{C}^{n\times n}$ be a square matrix (or a linear operator $T: V \to V$ on a vector space $V$ ). A scalar $\lambda \in \mathbb{C}$ is an eigenvalue of $A$ if there exists a nonzero vector $\mathbf{v} \in \mathbb{C}^n$ such that

1

\quad A\mathbf{v} = \lambda\mathbf{v}, \qquad \mathbf{v} \neq \mathbf{0}.

Derivation hint: This is the defining equation; the restriction $\mathbf{v}\neq\mathbf{0}$ is essential because $A\mathbf{0}=\lambda\mathbf{0}$ holds trivially for every $\lambda$ and would render the notion vacuous. Here $A$ is the $n\times n$ matrix, $\lambda$ is the eigenvalue (a scalar), $\mathbf{v}$ is the corresponding eigenvector, $I$ is the $n\times n$ identity matrix, and $\mathbf{0}$ is the zero vector. The vector $\mathbf{v}$ is said to belong to $\lambda$ .

DefinitionCharacteristic Polynomial, Spectrum, Eigenspace

The function $p_A(\lambda) = \det(A - \lambda I)$ is the characteristic polynomial of $A$ ; it is a degree- $n$ polynomial in $\lambda$ . Its roots are precisely the eigenvalues, and the multiset of roots is the spectrum $\sigma(A)$ . For each eigenvalue $\lambda$ , the set

3

\quad E_\lambda = \ker(A - \lambda I) = \{\mathbf{v} \in \mathbb{C}^n : A\mathbf{v} = \lambda\mathbf{v}\}

Derivation hint: The kernel of a linear map is always a subspace (closed under addition and scalar multiplication); $E_\lambda$ adds the zero vector to the eigenvectors to form the <strong>eigenspace</strong>. is a subspace called the eigenspace of $\lambda$ . Its dimension is the geometric multiplicity $g(\lambda) = \dim E_\lambda$ , while the multiplicity of $\lambda$ as a root of $p_A$ is the algebraic multiplicity $a(\lambda)$ .

TheoremMultiplicity Inequality and Diagonalizability

For every eigenvalue $\lambda$ of $A$ ,

4

\quad 1 \le g(\lambda) \le a(\lambda), \qquad \sum_{\lambda \in \sigma(A)} a(\lambda) = n.

Derivation hint: Geometric multiplicity is at least 1 because $\lambda$ being a root guarantees a nonzero kernel vector; the upper bound follows by extending an eigenbasis of $E_\lambda$ and computing the characteristic polynomial in block form. The sum equals $n$ over $\mathbb{C}$ by the Fundamental Theorem of Algebra. A matrix $A$ is diagonalizable (i.e. $A = PDP^{-1}$ with $D$ diagonal) if and only if $g(\lambda) = a(\lambda)$ for every eigenvalue, equivalently iff $A$ possesses $n$ linearly independent eigenvectors.

TheoremTrace and Determinant from Eigenvalues

Counting eigenvalues $\lambda_1, \dots, \lambda_n$ with algebraic multiplicity,

5

\quad \operatorname{tr}(A) = \sum_{i=1}^{n} \lambda_i, \qquad \det(A) = \prod_{i=1}^{n} \lambda_i.

Derivation hint: Expand $p_A(\lambda) = \prod_i(\lambda_i - \lambda)$ and match the coefficient of $\lambda^{n-1}$ (giving the trace) and the constant term $p_A(0)=\det A$ (giving the product).

ProofEigenvalues are roots of the characteristic polynomial

Step 1. Suppose $\lambda$ is an eigenvalue with eigenvector $\mathbf{v}\neq\mathbf{0}$ . From $[1]$ , $A\mathbf{v} - \lambda\mathbf{v} = \mathbf{0}$ , so $(A-\lambda I)\mathbf{v} = \mathbf{0}$ .

Step 2. Thus the homogeneous linear system with coefficient matrix $M = A - \lambda I$ has a nontrivial solution $\mathbf{v}$ . By the invertible matrix theorem, $M$ cannot be invertible.

Step 3. A square matrix is non-invertible iff its determinant vanishes. Hence $\det(A - \lambda I) = 0$ , which is equation $[2]$ .

Step 4. Conversely, if $\det(A-\lambda I)=0$ , then $A-\lambda I$ is singular, so its kernel contains some $\mathbf{v}\neq\mathbf{0}$ , which is then an eigenvector for $\lambda$ . Therefore the eigenvalues are exactly the roots of $p_A(\lambda)=\det(A-\lambda I)$ .

Step 5. Expanding the determinant, $p_A$ is a polynomial of degree exactly $n$ with leading term $(-1)^n\lambda^n$ . By the Fundamental Theorem of Algebra it has $n$ roots in $\mathbb{C}$ (with multiplicity), proving every complex square matrix has at least one eigenvalue. $\blacksquare$

∎

ProofEigenvectors for distinct eigenvalues are linearly independent

Step 1. Let $\mathbf{v}_1,\dots,\mathbf{v}_k$ be eigenvectors for distinct eigenvalues $\lambda_1,\dots,\lambda_k$ . Suppose, for contradiction, they are dependent; take a minimal dependent subset, reindexed so $c_1\mathbf{v}_1 + \cdots + c_m\mathbf{v}_m = \mathbf{0}$ with all $c_i\neq 0$ and $m$ smallest.

Step 2. Apply $A$ : $\sum_i c_i\lambda_i\mathbf{v}_i = \mathbf{0}$ . Also multiply the original relation by $\lambda_m$ : $\sum_i c_i\lambda_m\mathbf{v}_i = \mathbf{0}$ .

Step 3. Subtract: $\sum_{i=1}^{m-1} c_i(\lambda_i - \lambda_m)\mathbf{v}_i = \mathbf{0}$ . Since the $\lambda_i$ are distinct, each $(\lambda_i-\lambda_m)\neq 0$ , giving a shorter nontrivial dependence — contradicting minimality.

Step 4. Hence no dependence exists; the eigenvectors are linearly independent. A corollary: a matrix with $n$ distinct eigenvalues is automatically diagonalizable. $\blacksquare$

∎

DH

David Hilbert

1862–1943

Though eigenvalue ideas predate him, it was David Hilbert who, around 1904 in his work on integral equations, coined the German prefix eigen ("own," "characteristic," "proper"). He generalized the finite-dimensional spectral picture to infinite-dimensional function spaces, introducing what we now call Hilbert spaces and laying the spectral foundations that John von Neumann would later use to axiomatize quantum mechanics. Hilbert's insight — that an operator's "own values" govern the resonant frequencies of physical and analytic systems — turned eigenanalysis from a computational trick into a structural principle of modern mathematics and physics.

🧠 Intuition

Picture a linear map as a deformation of a rubber sheet: vectors get stretched, sheared, and rotated. Almost every arrow you draw will point in a new direction after the map. Eigenvectors are the special arrows that survive the deformation pointing the same way — they lie along the "grain" of the transformation. The eigenvalue tells you what happens along that grain: $\lambda > 1$ stretches, $0 < \lambda < 1$ compresses, $\lambda < 0$ flips, and $|\lambda| = 1$ preserves length. If you find enough independent grains to span the whole space, you can describe the map by saying simply "stretch by $\lambda_1$ along axis 1, by $\lambda_2$ along axis 2, …" — that is diagonalization. Complex eigenvalues $\lambda = re^{i\theta}$ signal that the map rotates by angle $\theta$ while scaling by $r$ ; there is then no real invariant line, which is why a 2D rotation has no real eigenvectors.

Figure

A shear matrix scaling its eigenvector while a generic vector turns

ExampleA symmetric 2×2 matrix

Given: $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ . Find its eigenvalues and eigenvectors.

Step 1 — Characteristic polynomial.

p_A(\lambda) = \det\begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix} = (2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3.

Step 2 — Roots. $\lambda^2 - 4\lambda + 3 = (\lambda-1)(\lambda-3) = 0 \Rightarrow \lambda_1 = 1,\ \lambda_2 = 3.$ Check: $\operatorname{tr} = 4 = 1+3$ ✓, $\det = 3 = 1\cdot 3$ ✓.

Step 3 — Eigenvector for $\lambda=1$ . Solve $(A - I)\mathbf{v} = \mathbf{0}$ : $\begin{pmatrix}1&1\\1&1\end{pmatrix}\mathbf{v}=\mathbf{0} \Rightarrow v_1 + v_2 = 0$ , so $\mathbf{v}_1 = (1, -1)^\top$ .

Step 4 — Eigenvector for $\lambda=3$ . $\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\mathbf{v}=\mathbf{0} \Rightarrow v_1 = v_2$ , so $\mathbf{v}_2 = (1,1)^\top$ . Note the eigenvectors are orthogonal (dot product $=0$ ), as guaranteed for symmetric matrices.

\boxed{\lambda_1 = 1,\ \mathbf{v}_1 = \begin{pmatrix}1\\-1\end{pmatrix}; \qquad \lambda_2 = 3,\ \mathbf{v}_2 = \begin{pmatrix}1\\1\end{pmatrix}}

ExampleComplex eigenvalues from a rotation-scaling matrix

Given: $A = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ , a 90° rotation. Find its eigenvalues.

Step 1 — Characteristic polynomial.

p_A(\lambda) = \det\begin{pmatrix} -\lambda & -1 \\ 1 & -\lambda \end{pmatrix} = \lambda^2 + 1.

Step 2 — Roots. $\lambda^2 = -1 \Rightarrow \lambda = \pm i.$ No real eigenvalues exist — geometrically obvious, since a rotation leaves no real line fixed.

Step 3 — Eigenvector for $\lambda = i$ . Solve $(A - iI)\mathbf{v}=\mathbf{0}$ : $\begin{pmatrix}-i&-1\\1&-i\end{pmatrix}\mathbf{v}=\mathbf{0}$ . From the second row $v_1 = i v_2$ , giving $\mathbf{v} = (i, 1)^\top$ .

Step 4 — Interpretation. $|\lambda| = 1$ confirms a pure rotation (no scaling); the argument $\arg(i) = 90°$ matches the rotation angle.

\boxed{\lambda = \pm i, \quad \mathbf{v}_{+i} = \begin{pmatrix} i \\ 1 \end{pmatrix}}

⚠️ Common mistake

"Every matrix is diagonalizable." False. A matrix is diagonalizable only when geometric and algebraic multiplicities agree for every eigenvalue (equation $[4]$ ). Matrices that fail this are called defective. Also beware: an eigenvector is a direction, never unique — any nonzero scalar multiple of $\mathbf{v}$ is also an eigenvector, so eigenspaces, not single vectors, are the invariant objects. Finally, $\det(A-\lambda I)=0$ is the characteristic equation; do not confuse it with solving $A\mathbf{v}=\mathbf{0}$ (that finds the null space, the eigenspace of $\lambda = 0$ ).

CounterexampleA defective matrix

Consider $A = \begin{pmatrix} 2 & 1 \\ 0 & 2 \end{pmatrix}$ . Its characteristic polynomial is $(2-\lambda)^2$ , so $\lambda = 2$ with algebraic multiplicity $a = 2$ . But solving $(A - 2I)\mathbf{v} = \begin{pmatrix}0&1\\0&0\end{pmatrix}\mathbf{v} = \mathbf{0}$ forces $v_2 = 0$ , giving the single eigenvector direction $(1,0)^\top$ . Hence $g(2) = 1 < 2 = a(2)$ : there are not enough eigenvectors to span $\mathbb{R}^2$ , so $A$ is not diagonalizable. The best one can do is the Jordan Canonical Form $\begin{pmatrix}2&1\\0&2\end{pmatrix}$ — a Jordan block.

connection

Eigen-analysis is the hub of linear algebra. It underlies Diagonalization (writing $A = PDP^{-1}$ ), generalizes to the Jordan Canonical Form when diagonalization fails, and refines into the Singular Value Decomposition for non-square or non-diagonalizable maps (via eigenvalues of $A^\top A$ ). For symmetric/Hermitian matrices, the Spectral Theorem guarantees a real spectrum and an orthonormal eigenbasis. The Matrix Exponential $e^{At} = Pe^{Dt}P^{-1}$ solves linear Systems of Differential Equations, while Markov Chains and PageRank hinge on the dominant eigenvalue. The characteristic polynomial ties eigenvalues to the Cayley–Hamilton Theorem, and to the Determinant and Trace through equation $[5]$ .

🔬 Where this appears

⏳ History

- 1743 — Euler. While solving systems of linear ODEs, Leonhard Euler encounters "characteristic" exponents, implicitly using eigenvalue ideas.

- 1760s — Lagrange. In studying the secular (long-term) motion of planetary orbits, Joseph-Louis Lagrange analyzes the "secular equation" — the characteristic equation in disguise.

- 1820s — Cauchy. Augustin-Louis Cauchy systematizes the theory of quadratic forms and proves that symmetric matrices have real "characteristic roots," an early Spectral Theorem.

- 1846 — Jacobi. Carl Jacobi develops iterative methods to diagonalize symmetric matrices, foreshadowing numerical eigenvalue algorithms.

- 1858 — Cayley. Arthur Cayley founds matrix algebra and (with Hamilton) the Cayley–Hamilton theorem linking a matrix to its characteristic polynomial.

- 1870s — Jordan & Weierstrass. Camille Jordan introduces the canonical form for defective matrices; Karl Weierstrass develops elementary divisors theory.

- 1904 — Hilbert. David Hilbert coins Eigenwert/*Eigenfunktion* and extends spectral theory to infinite dimensions in his work on integral equations.

- 1925–1932 — Heisenberg, von Neumann. Eigenvalues become the measured quantities of quantum mechanics; von Neumann gives the rigorous spectral theorem for unbounded operators.

- 1960s — Francis & Kublanovskaya. The QR algorithm is invented, making large-scale numerical eigenvalue computation practical and underpinning modern scientific computing.

open-problem

Numerical and theoretical frontiers. Computing eigenvalues of very large or sparse matrices remains demanding: the QR algorithm is robust but $O(n^3)$ , motivating ongoing research into Krylov-subspace methods (Lanczos, Arnoldi) and randomized algorithms. Eigenvalue conditioning is delicate — for non-normal matrices, eigenvalues can be wildly sensitive to perturbations, leading to the study of pseudospectra (Trefethen), crucial in fluid-dynamics stability where eigenvalues alone mislead. In infinite dimensions the spectrum may be continuous (no eigenvectors at all, as for the position operator), demanding the full machinery of functional analysis. Inverse eigenvalue problems (reconstruct a matrix/structure from prescribed spectra — "can you hear the shape of a drum?", answered no by Gordon–Webb–Wolpert in 1992) remain a rich, partly open area.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also