Hilbert Spaces

⚡ Key Takeaways

A Hilbert space is a complete inner product space — a vector space equipped with an inner product whose induced metric makes every Cauchy sequence converge. It is the natural setting where geometry (lengths, angles, orthogonality, projection) survives into infinite dimensions, and where Fourier-style expansions converge. Used in quantum mechanics, signal processing, PDEs, and probability. Named for David Hilbert; the abstract axioms were formalized by John von Neumann (1929–1932).

💡 The Core Insight

An inner product imports the full geometry of Euclidean space — orthogonality, the Pythagorean theorem, and orthogonal projection onto closed subspaces — into infinite dimensions. Completeness is the glue that makes infinite expansions $x=\sum c_n e_n$ actually converge to a point that exists. Together they guarantee a "best approximation" always exists, which is why Hilbert spaces dominate analysis.

DefinitionInner Product Space

Let $H$ be a vector space over a field $\mathbb{F}$ (either $\mathbb{R}$ or $\mathbb{C}$ ). An inner product is a map $\langle\cdot,\cdot\rangle: H\times H \to \mathbb{F}$ satisfying, for all $x,y,z\in H$ and $\alpha\in\mathbb{F}$ :

1

\quad \langle x,x\rangle \ge 0,\qquad \langle x,x\rangle = 0 \iff x = 0

Derivation hint: positive-definiteness — this is what makes $\|x\|=\sqrt{\langle x,x\rangle}$ a genuine norm rather than a seminorm.

2

\quad \langle \alpha x + z, y\rangle = \alpha\langle x,y\rangle + \langle z,y\rangle

Derivation hint: linearity in the first argument; combined with [3] this gives conjugate-bilinearity (the physics convention puts the conjugate-linear slot first).

3

\quad \langle x,y\rangle = \overline{\langle y,x\rangle}

Derivation hint: conjugate symmetry; over $\mathbb{R}$ the bar vanishes and the inner product is symmetric. Setting $y=x$ forces $\langle x,x\rangle\in\mathbb{R}$ , making [1] meaningful. Here $x,y,z$ are vectors, $\alpha$ a scalar, and $\overline{\;\cdot\;}$ denotes complex conjugation. The induced norm is $\|x\|=\sqrt{\langle x,x\rangle}$ .

DefinitionHilbert Space

A Hilbert space is an inner product space $(H,\langle\cdot,\cdot\rangle)$ that is complete with respect to the metric $d(x,y)=\|x-y\|$ — that is, every Cauchy sequence $(x_n)$ in $H$ converges to a limit $x\in H$ :

4

\quad \big(\forall \varepsilon>0\ \exists N\ \forall m,n\ge N:\ \|x_m-x_n\|<\varepsilon\big)\ \implies\ \exists x\in H:\ \|x_n-x\|\to 0

Derivation hint: completeness is a topological condition on the metric, not an algebraic one; an incomplete inner product space is called a pre-Hilbert space and its completion is a Hilbert space.

TheoremCauchy–Schwarz Inequality

For all $x,y$ in an inner product space,

5

\quad |\langle x,y\rangle| \le \|x\|\,\|y\|,

with equality if and only if $x$ and $y$ are linearly dependent. Derivation hint: for $\lambda = \langle x,y\rangle/\|y\|^2$ , expand $0\le \|x-\lambda y\|^2$ and simplify.

TheoremParallelogram Law

A normed space is an inner product space iff its norm satisfies

6

\quad \|x+y\|^2 + \|x-y\|^2 = 2\|x\|^2 + 2\|y\|^2.

Derivation hint: expand both sides using $\|u\|^2=\langle u,u\rangle$ ; the cross terms cancel. The converse (Jordan–von Neumann) recovers $\langle\cdot,\cdot\rangle$ via the polarization identity.

TheoremRiesz Representation Theorem

Every continuous linear functional $f: H\to\mathbb{F}$ on a Hilbert space is represented by a unique $y_f\in H$ :

7

\quad f(x) = \langle x, y_f\rangle\quad\text{for all }x\in H,\qquad \|f\| = \|y_f\|.

Derivation hint: if $f\equiv 0$ take $y_f=0$ ; otherwise project onto $(\ker f)^\perp$ , which is one-dimensional, and scale.

ProofCauchy–Schwarz via the Projection Trick

1. If $y=0$ both sides are $0$ ; assume $y\neq 0$ . 2. For any scalar $\lambda$ , positivity [1] gives

8

\quad 0 \le \|x-\lambda y\|^2 = \|x\|^2 - \overline\lambda\langle x,y\rangle - \lambda\overline{\langle x,y\rangle} + |\lambda|^2\|y\|^2.

Derivation hint: expand $\langle x-\lambda y, x-\lambda y\rangle$ using linearity [2] and conjugate symmetry [3]. 3. Choose $\lambda = \langle x,y\rangle/\|y\|^2$ to minimize the right side (this is the projection of $x$ onto $y$ ). 4. Substituting yields

9

\quad 0 \le \|x\|^2 - \frac{|\langle x,y\rangle|^2}{\|y\|^2},

Derivation hint: the two cross terms combine to $-2|\langle x,y\rangle|^2/\|y\|^2$ and the last term contributes $+|\langle x,y\rangle|^2/\|y\|^2$ . 5. Rearranging gives $|\langle x,y\rangle|^2 \le \|x\|^2\|y\|^2$ ; take square roots. Equality forces $x-\lambda y=0$ , i.e. linear dependence. $\blacksquare$

∎

ProofHilbert Projection Theorem (existence of best approximation)

Let $C\subseteq H$ be nonempty, closed, and convex, and $x\in H$ . We show a unique nearest point $p\in C$ exists. 1. Let $d=\inf_{c\in C}\|x-c\|$ and pick $c_n\in C$ with $\|x-c_n\|^2 \to d^2$ . 2. Apply the parallelogram law [6] to $u=x-c_m$ and $v=x-c_n$ :

10

\quad \|c_m-c_n\|^2 = 2\|x-c_m\|^2 + 2\|x-c_n\|^2 - 4\Big\|x-\tfrac{c_m+c_n}{2}\Big\|^2.

Derivation hint: this is [6] rearranged; the midpoint $\tfrac{c_m+c_n}{2}\in C$ by convexity, so its distance from $x$ is $\ge d$ . 3. Hence $\|c_m-c_n\|^2 \le 2\|x-c_m\|^2 + 2\|x-c_n\|^2 - 4d^2 \to 0$ , so $(c_n)$ is Cauchy. 4. By completeness [4], $c_n\to p$ , and since $C$ is closed, $p\in C$ with $\|x-p\|=d$ . 5. Uniqueness: if $p,p'$ both attain $d$ , step 2 with $c_m=p, c_n=p'$ forces $\|p-p'\|=0$ . $\blacksquare$ This is precisely where completeness is indispensable: without it the minimizing sequence need not converge inside the space.

∎

JV

John von Neumann

1903–1957

Von Neumann gave Hilbert spaces their modern axiomatic definition in 1929 and used them to put quantum mechanics on rigorous mathematical footing in his 1932 Mathematical Foundations of Quantum Mechanics. He unified Heisenberg's matrix mechanics and Schrödinger's wave mechanics by showing both are representations of operators on the same abstract separable Hilbert space — the matrices act on $\ell^2$ , the wavefunctions live in $L^2$ , and these spaces are isomorphic. His spectral theorem for unbounded self-adjoint operators is the backbone of observables and measurement in quantum theory.

🧠 Intuition

Think of a Hilbert space as $\mathbb{R}^n$ with infinitely many axes that you are allowed to use — provided your coordinates are "square-summable" so the vector has finite length. The inner product measures how much two vectors point in the same direction; orthogonality ( $\langle x,y\rangle=0$ ) means they share no direction at all. Any vector can be split into a part inside a closed subspace and a part perpendicular to it (orthogonal projection), exactly as a shadow on a wall plus the perpendicular gap. Completeness is the promise that if you keep adding more and more orthogonal pieces and the tail energy shrinks, the infinite sum genuinely lands on a vector that exists in the space — no points are "missing." This is why Fourier series of a function converge to something, and Hilbert space tells you that something is the original function.

Figure

Orthogonal projection of x onto a closed subspace M: the nearest point p and the perpendicular residual x−p

ExampleBest approximation in

\ell^2

Given: In $\ell^2$ , let $x=(1,\tfrac12,\tfrac13,\tfrac14,\dots)$ (entries $1/n$ , which is square-summable since $\sum 1/n^2 = \pi^2/6$ ). Let $M=\overline{\operatorname{span}}\{e_1,e_2\}$ , the closed subspace of sequences supported on the first two coordinates. Find the orthogonal projection $p$ and the residual norm.

Steps: 1. Projection onto coordinate axes just keeps those coordinates:

11

\quad p = \langle x,e_1\rangle e_1 + \langle x,e_2\rangle e_2 = 1\cdot e_1 + \tfrac12\cdot e_2 = (1,\tfrac12,0,0,\dots).

Derivation hint: $\{e_1,e_2\}$ is orthonormal, so $\operatorname{proj}_M x = \sum_k \langle x,e_k\rangle e_k$ . 2. Residual $x-p = (0,0,\tfrac13,\tfrac14,\dots)$ . 3. Residual energy:

12

\quad \|x-p\|^2 = \sum_{n=3}^\infty \frac1{n^2} = \frac{\pi^2}{6} - 1 - \frac14 = \frac{\pi^2}{6} - \frac54.

Derivation hint: total energy $\sum 1/n^2 = \pi^2/6$ minus the captured terms $1^2+(1/2)^2$ .

Answer: $\boxed{p=(1,\tfrac12,0,0,\dots),\quad \|x-p\| = \sqrt{\tfrac{\pi^2}{6}-\tfrac54}\approx 0.3434}$ (dimensionless).

⚠️ Common mistake

Completeness is not optional, and not every inner product space has it. The space $C[0,1]$ of continuous functions with the $L^2$ inner product is an inner product space but is not a Hilbert space — Cauchy sequences of continuous functions can converge (in $L^2$ ) to discontinuous limits outside the space. Its completion is $L^2[0,1]$ , which is a Hilbert space. Also beware: a "basis" of a Hilbert space (an orthonormal basis / Schauder basis) is generally not a Hamel (algebraic) basis — infinite-dimensional Hilbert spaces have uncountable Hamel bases, but separable ones have countable orthonormal bases where convergence is in norm, not finite sums.

CounterexampleNot every norm comes from an inner product

Consider $\mathbb{R}^2$ with the sup norm $\|(a,b)\|_\infty = \max(|a|,|b|)$ . Take $x=(1,0)$ , $y=(0,1)$ . Then $\|x+y\|_\infty^2 + \|x-y\|_\infty^2 = 1+1 = 2$ , but $2\|x\|^2 + 2\|y\|^2 = 2+2 = 4$ . The parallelogram law [6] fails, so no inner product induces this norm — $(\mathbb{R}^2,\|\cdot\|_\infty)$ is a Banach space but not a Hilbert space. The same holds for every $\ell^p$ and $L^p$ with $p\neq 2$ : only $p=2$ is Hilbert.

connection

Hilbert spaces are the geometrically richest members of the family of Banach Spaces (complete normed spaces); the extra inner-product structure adds orthogonality and self-duality (via Riesz Representation Theorem) that general Banach spaces lack. They are the natural domain of Bounded Linear Operators and the setting for the Spectral Theorem, which diagonalizes self-adjoint operators just as symmetric matrices are diagonalized in Linear Algebra. Convergence of Fourier Series is best understood as orthonormal expansion in $L^2$ . The completion construction relates them to Metric Space Completion, and the prototype $\ell^2$ links them to Sequence Spaces.

🔬 Where this appears

⏳ History

- 1900–1910: David Hilbert studies integral equations and the space of square-summable sequences

\ell^2

, introducing infinitely many variables; the geometry is implicit, not yet axiomatized.

- 1907: Frigyes Riesz and Ernst Fischer prove the Riesz–Fischer theorem, establishing that

L^2

is complete and isomorphic to

\ell^2

— the first deep "uniqueness" result.

- 1907: Riesz proves the representation theorem for continuous linear functionals on

L^2

.

- 1918: Erhard Schmidt and Riesz develop the geometry of

\ell^2

— orthogonality, projection — making the Euclidean analogy explicit.

- 1922: Stefan Banach axiomatizes complete normed spaces (Banach spaces) in his thesis, providing the broader framework.

- 1929: John von Neumann gives the first abstract axiomatic definition of "Hilbert space" (complete, separable, infinite-dimensional inner product space) in his work on quantum mechanics.

- 1932: Von Neumann publishes Mathematische Grundlagen der Quantenmechanik, proving the spectral theorem for unbounded self-adjoint operators and unifying matrix and wave mechanics.

- 1934: Jordan and von Neumann prove the parallelogram law characterization — exactly which norms come from inner products.

- 1930s–40s: The separability requirement is dropped; general (possibly non-separable) Hilbert spaces become standard, and the theory of operator algebras (von Neumann algebras) grows out of it.

open-problem

The invariant subspace problem remains open for separable Hilbert spaces: does every bounded linear operator on an infinite-dimensional separable Hilbert space have a nontrivial closed invariant subspace? Per Enflo (1976/1987) and Charles Read constructed counterexamples on certain Banach spaces, but the Hilbert space case is unresolved — a striking reminder that even in the most structured infinite-dimensional setting, operator theory hides deep mysteries. Related subtleties: in infinite dimensions the closed unit ball is not compact (Riesz's lemma), so existence arguments must use weak compactness or completeness rather than the finite-dimensional Heine–Borel intuition; and bounded operators need not attain their norm.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also