Eigenanalysis

⚡ Key Takeaways

Eigenanalysis is the study of special directions (eigenvectors) along which a linear transformation acts purely by scaling (by an eigenvalue). One-line statement: for a square matrix $A$ , find scalars $\lambda$ and nonzero vectors $\mathbf{v}$ with $A\mathbf{v}=\lambda\mathbf{v}$ . Key result: when $A$ has a full set of independent eigenvectors it can be diagonalized, making powers, exponentials, and dynamics trivial to compute. Where it applies: differential equations, vibrations, quantum mechanics, PCA, PageRank, stability analysis. Who/when: roots in Euler (1750s) and Lagrange, formalized by Cauchy (1820s), named "eigenvalue" via Hilbert (~1904).

💡 The Core Insight

A linear transformation is fully understood once you know how it acts on a basis. The best basis is one of eigenvectors, because in that basis the matrix becomes diagonal: every coordinate evolves independently, scaled by its own eigenvalue. Eigenanalysis is the search for that privileged coordinate system. Complicated coupled behavior is just simple independent behavior viewed through a tilted lens.

DefinitionEigenvalue and Eigenvector

Let $A$ be an $n\times n$ matrix over a field $\mathbb{F}$ (typically $\mathbb{R}$ or $\mathbb{C}$ ). A scalar $\lambda\in\mathbb{F}$ is an eigenvalue of $A$ if there exists a nonzero vector $\mathbf{v}\in\mathbb{F}^n$ , called an eigenvector, satisfying

1

\quad A\mathbf{v}=\lambda\mathbf{v},\qquad \mathbf{v}\neq\mathbf{0}.

Derivation hint: this is the defining relation — the map $A$ acts on $\mathbf{v}$ purely as multiplication by the scalar $\lambda$ . Here $A$ is the matrix (the linear map), $\mathbf{v}$ is the invariant direction, and $\lambda$ is the stretch factor. The eigenspace of $\lambda$ is $E_\lambda=\ker(A-\lambda I)$ , the set of all eigenvectors for $\lambda$ together with $\mathbf{0}$ .

DefinitionCharacteristic Polynomial

Rearranging [1] gives $(A-\lambda I)\mathbf{v}=\mathbf{0}$ . A nonzero solution exists iff $A-\lambda I$ is singular:

2

\quad p_A(\lambda)=\det(A-\lambda I)=0.

Derivation hint: a homogeneous system $M\mathbf{v}=\mathbf{0}$ has a nontrivial solution iff $\det M=0$ ; set $M=A-\lambda I$ . $p_A(\lambda)$ is the characteristic polynomial, of degree $n$ in $\lambda$ . Its roots are exactly the eigenvalues. $I$ is the $n\times n$ identity.

TheoremSpectral Theorem for Symmetric Matrices

If $A$ is a real symmetric matrix ( $A=A^{\mathsf T}$ ), then all eigenvalues are real, and there exists an orthonormal basis of $\mathbb{R}^n$ consisting of eigenvectors of $A$ . Equivalently, there is an orthogonal matrix $Q$ ( $Q^{\mathsf T}Q=I$ ) and a diagonal $\Lambda$ with

3

\quad A=Q\Lambda Q^{\mathsf T},\qquad \Lambda=\operatorname{diag}(\lambda_1,\dots,\lambda_n).

Derivation hint: combine reality of eigenvalues, orthogonality of eigenvectors for distinct eigenvalues, and Gram–Schmidt within repeated eigenspaces.

TheoremDiagonalizability Criterion

An $n\times n$ matrix $A$ is diagonalizable iff it has $n$ linearly independent eigenvectors. Equivalently, for every eigenvalue the geometric multiplicity equals the algebraic multiplicity. Then

4

\quad A=P\Lambda P^{-1},\qquad P=[\mathbf{v}_1\,|\,\cdots\,|\,\mathbf{v}_n],\quad \Lambda=\operatorname{diag}(\lambda_1,\dots,\lambda_n).

Derivation hint: writing $AP=P\Lambda$ column by column reproduces $A\mathbf{v}_i=\lambda_i\mathbf{v}_i$ ; invertibility of $P$ requires independent eigenvectors.

ProofFrom the Eigen-equation to Diagonalization

Step 1. Start from [1]: $A\mathbf{v}=\lambda\mathbf{v}$ for each eigenpair $(\lambda_i,\mathbf{v}_i)$ , $i=1,\dots,n$ .

Step 2. Stack the eigenvectors as columns of $P=[\mathbf{v}_1\,|\,\cdots\,|\,\mathbf{v}_n]$ . Multiplying $A$ by $P$ acts on each column:

5

\quad AP=[A\mathbf{v}_1\,|\,\cdots\,|\,A\mathbf{v}_n]=[\lambda_1\mathbf{v}_1\,|\,\cdots\,|\,\lambda_n\mathbf{v}_n].

Derivation hint: matrix–matrix product is column-wise application of $A$ , then substitute [1] into each column.

Step 3. The right side equals $P\Lambda$ , since scaling column $i$ of $P$ by $\lambda_i$ is post-multiplication by the diagonal matrix $\Lambda$ :

6

\quad AP=P\Lambda.

Derivation hint: $(P\Lambda)_{\cdot i}=\lambda_i\,(P)_{\cdot i}$ because $\Lambda$ is diagonal.

Step 4. If the $\mathbf{v}_i$ are linearly independent then $P$ is invertible. Right-multiply [6] by $P^{-1}$ :

7

\quad A=P\Lambda P^{-1},

Derivation hint: multiply both sides of [6] on the right by $P^{-1}$ ; this isolates $A$ . establishing [4]. Step 5. Powers become trivial: $A^k=P\Lambda^k P^{-1}$ , because adjacent $P^{-1}P=I$ cancel:

8

\quad A^k=P\,\operatorname{diag}(\lambda_1^k,\dots,\lambda_n^k)\,P^{-1}.

Derivation hint: $A^k=(P\Lambda P^{-1})^k$ ; the inner $P^{-1}P$ pairs collapse, leaving $\Lambda^k$ . $\blacksquare$

∎

Figure

Action of a matrix on the unit circle: eigen-directions are mapped onto themselves

AC

Augustin-Louis Cauchy

1789–1857

Cauchy gave the first systematic treatment of what we now call eigenvalues while studying the principal axes of quadric surfaces and the motion of rigid bodies. In an 1829 memoir he proved that the eigenvalues of a real symmetric matrix are real — the cornerstone of the Spectral Theorem. His work on the "secular equation" (so named because it arose in computing slow, secular, perturbations of planetary orbits) connected algebra to celestial mechanics and established the determinant condition [2] as the gateway to the entire theory.

🧠 Intuition

Think of a matrix as a machine that takes vectors in and pushes vectors out, generally rotating and stretching them. Most input directions come out pointing somewhere new. But a few special directions — the eigenvectors — come out pointing the same way (or exactly reversed), only longer or shorter. The eigenvalue tells you the stretch factor along that direction. If you set up your coordinate axes along these special directions, the machine stops mixing your coordinates: each axis simply scales by its eigenvalue. That is why eigenanalysis turns hard coupled problems into easy independent ones — it finds the "natural axes" of the transformation.

ExampleA 2×2 symmetric matrix

Given: $A=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ .

Step 1 — characteristic polynomial (use [2]):

\det\!\begin{pmatrix}2-\lambda&1\\1&2-\lambda\end{pmatrix}=(2-\lambda)^2-1=\lambda^2-4\lambda+3.

Step 2 — roots: $\lambda^2-4\lambda+3=(\lambda-1)(\lambda-3)=0$ , so $\lambda_1=3$ , $\lambda_2=1$ .

Step 3 — eigenvectors. For $\lambda=3$ : $(A-3I)\mathbf{v}=\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\mathbf{v}=0\Rightarrow \mathbf{v}_1=(1,1)^{\mathsf T}$ . For $\lambda=1$ : $\begin{pmatrix}1&1\\1&1\end{pmatrix}\mathbf{v}=0\Rightarrow \mathbf{v}_2=(1,-1)^{\mathsf T}$ .

Step 4 — orthonormalize (symmetric ⇒ orthogonal eigenvectors): $\hat{\mathbf v}_1=\tfrac1{\sqrt2}(1,1)$ , $\hat{\mathbf v}_2=\tfrac1{\sqrt2}(1,-1)$ .

\boxed{\;\lambda_1=3,\ \mathbf v_1=(1,1);\quad \lambda_2=1,\ \mathbf v_2=(1,-1)\;}

(Dimensionless: these are pure scaling factors and direction vectors.)

ExampleComputing a matrix power via diagonalization

Given: $A=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ as above; compute $A^{10}$ applied to $\mathbf x_0=(1,0)^{\mathsf T}$ .

Step 1 — expand $\mathbf x_0$ in the eigenbasis: $(1,0)=\tfrac12(1,1)+\tfrac12(1,-1)=\tfrac12\mathbf v_1+\tfrac12\mathbf v_2$ .

Step 2 — apply [8]: each eigencomponent scales by $\lambda^{10}$ :

A^{10}\mathbf x_0=\tfrac12(3^{10})\mathbf v_1+\tfrac12(1^{10})\mathbf v_2.

Step 3 — numbers: $3^{10}=59049$ , $1^{10}=1$ .

A^{10}\mathbf x_0=\tfrac{59049}{2}(1,1)+\tfrac12(1,-1)=(29525,\;29524).

\boxed{\;A^{10}(1,0)^{\mathsf T}=(29525,\,29524)^{\mathsf T}\;}

Note how the $\lambda=3$ direction dominates — the slow $\lambda=1$ part is negligible after many iterations, illustrating why the largest eigenvalue governs long-term behavior.

⚠️ Common mistake

Eigenvectors are not unique, but eigen-directions are. Any nonzero scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue. Normalization (e.g. unit length) is a convention, not a property of $A$ . Conversely, every matrix has eigenvalues over $\mathbb{C}$ (fundamental theorem of algebra applied to [2]), but a real matrix may have no real eigenvectors — its eigenvalues can be complex conjugate pairs encoding rotation. Also: the determinant being zero in [2] is a condition on $\lambda$ , not a claim that $A$ is singular. $A$ is singular iff $0$ is one of its eigenvalues.

CounterexampleA non-diagonalizable matrix

The shear $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ has characteristic polynomial $(1-\lambda)^2$ , so $\lambda=1$ with algebraic multiplicity 2. But $(A-I)=\begin{pmatrix}0&1\\0&0\end{pmatrix}$ has rank 1, so its kernel is one-dimensional: only one independent eigenvector $(1,0)^{\mathsf T}$ . Geometric multiplicity $1<2=$ algebraic multiplicity. By [4]'s criterion, $A$ is not diagonalizable — it is defective. Such matrices require the Jordan Normal Form instead. This shows the diagonalization theorem genuinely needs its hypothesis.

connection

Eigenanalysis sits at a crossroads of linear algebra. The Singular Value Decomposition generalizes it to non-square matrices using eigenvalues of $A^{\mathsf T}A$ . The Spectral Theorem is its cleanest case (symmetric/Hermitian matrices). For defective matrices it gives way to the Jordan Normal Form. The Characteristic Polynomial links it to abstract algebra via the Cayley–Hamilton Theorem, which states every matrix satisfies its own characteristic equation. Eigenvalues also determine the Matrix Exponential $e^{At}$ , the engine of Linear Differential Equations, and the spectral radius controls convergence of Iterative Methods.

🔬 Where this appears

⏳ History

- 1750s — Euler & Lagrange: principal axes of inertia of rotating bodies lead to the implicit eigenvalue problem; Euler studies quadric surfaces.

- 1760s — Lagrange: systems of coupled oscillations (small vibrations) produce the "secular equation," the determinant condition [2] in disguise.

- 1820s — Cauchy: systematic theory of determinants; proves real symmetric matrices have real eigenvalues (1829), and coins the link to quadratic forms.

- 1840s — Jacobi: develops iterative diagonalization of symmetric matrices (the Jacobi method), still used numerically.

- 1858 — Cayley: A Memoir on the Theory of Matrices formalizes matrix algebra; states the Cayley–Hamilton theorem for

2\times2

and

3\times3

.

- 1878 — Frobenius: rigorous general proof of Cayley–Hamilton; theory of minimal polynomials.

- 1870 — Jordan: introduces the canonical (Jordan) form handling defective matrices.

- ~1904 — Hilbert: in his work on integral equations introduces the German term Eigenwert ("proper/own value"), giving us "eigenvalue."

- 1929–1965 — von Neumann, Wilkinson, Francis: spectral theory for operators in quantum mechanics, then the QR algorithm (Francis, 1961) — the standard modern numerical eigensolver.

open-problem

Several practical and theoretical issues remain delicate. Numerical conditioning: eigenvalues of nonsymmetric matrices can be extremely sensitive to tiny perturbations (the field of pseudospectra studies this); a matrix may be "almost defective" so that computed eigenvectors are nearly parallel and ill-conditioned. Large-scale spectra: for matrices of dimension $10^6$ and beyond, only a few extremal eigenvalues are accessible (Krylov / Lanczos / Arnoldi methods), and rigorous error bounds for interior eigenvalues are subtle. Random matrix theory asks for the distribution of eigenvalues of large random matrices (Wigner semicircle, Tracy–Widom edge laws) — connections to number theory (the Riemann zeta zeros conjecturally follow GUE statistics) remain open. Finally, non-normal operators in infinite dimensions can have empty or pathological spectra, where the finite-dimensional intuition fully breaks down.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also