Hilbert Spaces and Inner Products

⚡ Key Takeaways

A Hilbert space is a complete inner product space — a vector space equipped with a notion of length and angle (the inner product) in which every Cauchy sequence converges. Key result: such spaces possess orthonormal bases and the geometry of projection (the Projection Theorem), generalizing finite-dimensional Euclidean geometry to infinite dimensions. Applies in quantum mechanics, Fourier analysis, signal processing, and PDE theory. Named for David Hilbert (1862–1943); abstracted by von Neumann (1929).

💡 The Core Insight

The inner product turns an abstract vector space into a geometric object: you can measure how much one vector points along another. Completeness then guarantees that limits exist where geometry says they should. Together these give the single most powerful tool — orthogonal projection — letting you decompose any vector into a part inside a subspace and a part perpendicular to it, even when the subspace is infinite-dimensional.

Formal Statement

DefinitionInner Product

Let HH be a vector space over the field F\mathbb{F} (either R\mathbb{R} or C\mathbb{C}). An inner product is a map ⟨⋅,⋅⟩:H×H→F\langle\cdot,\cdot\rangle : H \times H \to \mathbb{F} satisfying, for all x,y,z∈Hx,y,z \in H and α∈F\alpha \in \mathbb{F}:

  • Conjugate symmetry: ⟨x,y⟩=⟨y,x⟩‾\langle x,y\rangle = \overline{\langle y,x\rangle}
  • Linearity in the first argument: ⟨αx+z,y⟩=α⟨x,y⟩+⟨z,y⟩\langle \alpha x + z, y\rangle = \alpha\langle x,y\rangle + \langle z,y\rangle
  • Positive-definiteness: ⟨x,x⟩≥0\langle x,x\rangle \ge 0, with equality iff x=0x = 0

Here (⋅)‾\overline{(\cdot)} denotes complex conjugation (trivial in the real case). The induced norm is ∥x∥=⟨x,x⟩\|x\| = \sqrt{\langle x,x\rangle}, and the distance is d(x,y)=∥x−y∥d(x,y) = \|x-y\|.

1
∥x∥=⟨x,x⟩,d(x,y)=∥x−y∥\quad \|x\| = \sqrt{\langle x,x\rangle}, \qquad d(x,y) = \|x-y\|

Derivation hint: positive-definiteness guarantees ⟨x,x⟩≥0\langle x,x\rangle \ge 0, so the square root is well-defined and real.

⚠️ Convention on linearity

Mathematicians often take linearity in the first argument (physicists, via Dirac notation, take it in the second). The two differ only by which slot carries the conjugate. Throughout this article linearity is in the first slot, so ⟨x,αy⟩=α‾⟨x,y⟩\langle x, \alpha y\rangle = \overline{\alpha}\langle x,y\rangle (conjugate-linearity in the second slot).

DefinitionHilbert Space

A Hilbert space is an inner product space (H,⟨⋅,⋅⟩)(H, \langle\cdot,\cdot\rangle) that is complete with respect to the metric d(x,y)=∥x−y∥d(x,y)=\|x-y\|: every Cauchy sequence (xn)(x_n) in HH converges to a limit in HH. An incomplete inner product space is called a pre-Hilbert space; its completion is a Hilbert space.

TheoremCauchy–Schwarz Inequality

For all x,yx,y in an inner product space,

2
∣⟨x,y⟩∣≤∥x∥ ∥y∥,\quad |\langle x,y\rangle| \le \|x\|\,\|y\|,

with equality if and only if xx and yy are linearly dependent.

Derivation hint: minimize ∥x−ty∥2≥0\|x - t y\|^2 \ge 0 over the scalar tt; the optimal tt yields the inequality.

TheoremParallelogram Law characterizes inner-product norms

A norm ∥⋅∥\|\cdot\| on a vector space arises from an inner product if and only if it satisfies

3
∥x+y∥2+∥x−y∥2=2∥x∥2+2∥y∥2.\quad \|x+y\|^2 + \|x-y\|^2 = 2\|x\|^2 + 2\|y\|^2.

When it does, the inner product is recovered by the polarization identity (complex case):

4
⟨x,y⟩=14∑k=03ik ∥x+iky∥2.\quad \langle x,y\rangle = \tfrac14\sum_{k=0}^{3} i^{k}\,\|x + i^{k} y\|^2.

Derivation hint: expand each ∥x±y∥2=⟨x±y,x±y⟩\|x \pm y\|^2 = \langle x\pm y, x\pm y\rangle and add; polarization comes from forming the same combination of the four norm expansions.

TheoremProjection Theorem

Let MM be a closed subspace of a Hilbert space HH. Then every x∈Hx \in H has a unique decomposition

5
x=m+n,m∈M, n∈M⊥,\quad x = m + n, \qquad m \in M,\ n \in M^\perp,

where M⊥={y∈H:⟨y,m⟩=0 ∀m∈M}M^\perp = \{ y \in H : \langle y, m\rangle = 0 \ \forall m \in M\}. The vector m=PMxm = P_M x is the unique closest point of MM to xx, and H=M⊕M⊥H = M \oplus M^\perp.

Derivation hint: minimize ∥x−m∥\|x - m\| over m∈Mm \in M; completeness plus the parallelogram law makes the minimizing sequence Cauchy, hence convergent, giving the orthogonal component.

Why It Holds / Derivation

ProofProjection Theorem

Step 1 — Set up the distance. Let δ=inf⁡m∈M∥x−m∥\delta = \inf_{m\in M}\|x - m\|. Choose a sequence mk∈Mm_k \in M with ∥x−mk∥→δ\|x - m_k\| \to \delta.

Step 2 — Show the sequence is Cauchy. Apply the parallelogram law [3] to the vectors x−mjx - m_j and x−mkx - m_k:

6
∥mj−mk∥2=2∥x−mj∥2+2∥x−mk∥2−4∥x−mj+mk2∥2.\quad \|m_j - m_k\|^2 = 2\|x-m_j\|^2 + 2\|x-m_k\|^2 - 4\left\|x - \tfrac{m_j+m_k}{2}\right\|^2.

Derivation hint: substitute u=x−mju = x-m_j, v=x−mkv = x-m_k into [3]; note u−v=mk−mju-v = m_k-m_j and u+v=2x−(mj+mk)u+v = 2x-(m_j+m_k). Since mj+mk2∈M\tfrac{m_j+m_k}{2} \in M (subspace), the last term is ≥δ2\ge \delta^2. As j,k→∞j,k\to\infty the first two terms →2δ2+2δ2\to 2\delta^2 + 2\delta^2, so ∥mj−mk∥2→0\|m_j-m_k\|^2 \to 0. Hence (mk)(m_k) is Cauchy.

Step 3 — Use completeness. HH complete and MM closed imply mk→m∈Mm_k \to m \in M with ∥x−m∥=δ\|x-m\| = \delta.

Step 4 — Orthogonality. Set n=x−mn = x - m. For any w∈Mw \in M and scalar tt, ∥n−tw∥2≥δ2=∥n∥2\|n - t w\|^2 \ge \delta^2 = \|n\|^2. Expanding:

7
−2 Re(tˉ ⟨n,w⟩)+∣t∣2∥w∥2≥0∀t.\quad -2\,\mathrm{Re}\big(\bar t\,\langle n, w\rangle\big) + |t|^2\|w\|^2 \ge 0 \quad \forall t.

Derivation hint: ∥n−tw∥2=∥n∥2−2Re(tˉ⟨n,w⟩)+∣t∣2∥w∥2\|n-tw\|^2 = \|n\|^2 - 2\mathrm{Re}(\bar t\langle n,w\rangle) + |t|^2\|w\|^2; subtract ∥n∥2\|n\|^2. Choosing t=s⟨n,w⟩t = s\langle n,w\rangle for small real s>0s>0 forces ⟨n,w⟩=0\langle n,w\rangle = 0. Thus n∈M⊥n \in M^\perp.

Step 5 — Uniqueness. If x=m1+n1=m2+n2x = m_1+n_1 = m_2+n_2 with mi∈Mm_i\in M, ni∈M⊥n_i\in M^\perp, then m1−m2=n2−n1∈M∩M⊥m_1-m_2 = n_2-n_1 \in M\cap M^\perp. But any v∈M∩M⊥v\in M\cap M^\perp satisfies ⟨v,v⟩=0\langle v,v\rangle = 0, so v=0v=0. ■\blacksquare

∎
TheoremRiesz Representation Theorem

Every continuous linear functional f:H→Ff : H \to \mathbb{F} on a Hilbert space is given by inner product against a unique vector yf∈Hy_f \in H:

8
f(x)=⟨x,yf⟩∀x∈H,∥f∥=∥yf∥.\quad f(x) = \langle x, y_f\rangle \quad \forall x\in H, \qquad \|f\| = \|y_f\|.

Derivation hint: if f=0f=0 take yf=0y_f=0; otherwise ker⁡f\ker f is closed with one-dimensional complement spanned by some z∈(ker⁡f)⊥z\in(\ker f)^\perp, and yf=f(z)‾∥z∥2 zy_f = \dfrac{\overline{f(z)}}{\|z\|^2}\,z.

JV
John von Neumann
1903–1957

Hilbert spaces are named for Hilbert, but their axiomatic definition is due to John von Neumann, who in his 1929–1932 work on the mathematical foundations of quantum mechanics isolated exactly the properties needed: a complex inner product space that is complete and (in his treatment) separable. He recognized that the states of a quantum system are unit vectors in such a space and that observables are self-adjoint operators on it. This unified Heisenberg's matrix mechanics (ℓ2\ell^2) and Schrödinger's wave mechanics (L2L^2) by showing both are the same Hilbert space in disguise — a striking confirmation of Riesz–Fischer Theorem in physics.

Building Intuition

🧠 Intuition

Think of a Hilbert space as infinite-dimensional Euclidean space that never has holes. The inner product gives you a protractor and ruler: ⟨x,y⟩=0\langle x,y\rangle = 0 means perpendicular, ⟨x,y⟩/(∥x∥∥y∥)\langle x,y\rangle/(\|x\|\|y\|) is the cosine of the angle. The norm is straight-line length. Completeness is the promise that if a sequence of vectors looks like it should converge (terms bunch arbitrarily close together), then it genuinely converges to a vector still inside the space — no point is missing. The single most useful mental picture is the drop-a-perpendicular operation: to approximate a function by, say, a trigonometric polynomial, you orthogonally project it onto the subspace of such polynomials. The Fourier coefficients are literally the components of that projection. "Best approximation" is geometric, not just analytic.

Figure
Orthogonal projection of x onto a closed subspace M: the residual n = x − P_M x is perpendicular to M
M x m = P_M x n = x − P_M x ⊥ M 0

Worked Examples

:::example[Fourier coefficient as a projection in L2[−π,π]L^2[-\pi,\pi]] Given: Work in H=L2[−π,π]H = L^2[-\pi,\pi] with ⟨f,g⟩=∫−ππf(t)g(t)‾ dt\langle f,g\rangle = \int_{-\pi}^{\pi} f(t)\overline{g(t)}\,dt. The functions en(t)=12πeinte_n(t) = \tfrac{1}{\sqrt{2\pi}}e^{int}, n∈Zn\in\mathbb{Z}, form an orthonormal set. Find the best L2L^2 approximation of f(t)=tf(t) = t by the single mode e1e_1, and its coefficient.

Step 1 — Orthonormality check. ⟨en,em⟩=12π∫−ππei(n−m)tdt=δnm\langle e_n, e_m\rangle = \tfrac{1}{2\pi}\int_{-\pi}^{\pi} e^{i(n-m)t}dt = \delta_{nm}. ✓

Step 2 — Coefficient = projection onto e1e_1.

c1=⟨f,e1⟩=12π∫−ππt e−it dt.c_1 = \langle f, e_1\rangle = \frac{1}{\sqrt{2\pi}}\int_{-\pi}^{\pi} t\,e^{-it}\,dt.

Step 3 — Evaluate the integral. Integrate by parts: ∫t e−itdt=(it+1)e−it\int t\,e^{-it}dt = (it+1)e^{-it}. Evaluate from −π-\pi to π\pi:

∫−ππte−itdt=(iπ+1)e−iπ−(−iπ+1)eiπ=−(iπ+1)−(−(−iπ+1))=−2πi.\int_{-\pi}^{\pi} t e^{-it}dt = (i\pi+1)e^{-i\pi} - (-i\pi+1)e^{i\pi} = -(i\pi+1) - (-(-i\pi+1)) = -2\pi i.

So c1=−2πi2π=−i2πc_1 = \dfrac{-2\pi i}{\sqrt{2\pi}} = -i\sqrt{2\pi}.

Step 4 — Best approximation. The projection is c1e1=−i2π⋅12πeit=−i eitc_1 e_1 = -i\sqrt{2\pi}\cdot \tfrac{1}{\sqrt{2\pi}}e^{it} = -i\,e^{it}.

Answer:

  c1=−i2π,Pe1f=−i eit  \boxed{\;c_1 = -i\sqrt{2\pi}, \qquad P_{e_1} f = -i\,e^{it}\;}

(In real-form, −ieit=sin⁡t−icos⁡t-i e^{it} = \sin t - i\cos t; the real Fourier sine coefficient b1=2b_1 = 2 matches t≈2sin⁡tt\approx 2\sin t.)

ExampleClosest point in ℓ2\ell^2 to a finite-dimensional subspace

Given: In ℓ2={(x1,x2,… ):∑∣xk∣2<∞}\ell^2 = \{(x_1,x_2,\dots): \sum |x_k|^2 < \infty\} with ⟨x,y⟩=∑kxkyˉk\langle x,y\rangle = \sum_k x_k\bar y_k, let x=(1, 2, 13, 14, 15,… )x = (1,\,2,\,\tfrac13,\,\tfrac14,\,\tfrac15,\dots) where xk=1/kx_k = 1/k for k≥3k\ge 3, and x1=1x_1=1, x2=2x_2=2. Let M=span{e1,e2}M = \mathrm{span}\{e_1,e_2\} (first two coordinate vectors). Find PMxP_M x and the distance ∥x−PMx∥\|x - P_M x\|.

Step 1 — Verify x∈ℓ2x\in\ell^2. ∥x∥2=1+4+∑k≥31k2\|x\|^2 = 1 + 4 + \sum_{k\ge3}\tfrac1{k^2}. Since ∑k≥31k2=π26−1−14=π26−1.25≈0.3949\sum_{k\ge3}\tfrac1{k^2} = \tfrac{\pi^2}{6}-1-\tfrac14 = \tfrac{\pi^2}{6}-1.25 \approx 0.3949, finite. ✓

Step 2 — Project. Since e1,e2e_1,e_2 orthonormal: PMx=⟨x,e1⟩e1+⟨x,e2⟩e2=1⋅e1+2⋅e2=(1,2,0,0,… )P_M x = \langle x,e_1\rangle e_1 + \langle x,e_2\rangle e_2 = 1\cdot e_1 + 2\cdot e_2 = (1,2,0,0,\dots).

Step 3 — Residual. x−PMx=(0,0,13,14,… )x - P_M x = (0,0,\tfrac13,\tfrac14,\dots), the "tail."

Step 4 — Distance.

∥x−PMx∥2=∑k≥31k2=π26−1−14≈0.3949.\|x-P_M x\|^2 = \sum_{k\ge3}\frac{1}{k^2} = \frac{\pi^2}{6} - 1 - \frac14 \approx 0.3949.

Answer:

  PMx=(1,2,0,0,… ),dist=π26−1.25≈0.6284  \boxed{\;P_M x = (1,2,0,0,\dots), \qquad \mathrm{dist} = \sqrt{\tfrac{\pi^2}{6}-1.25}\approx 0.6284\;}

Common Misconceptions

⚠️ Every norm is NOT an inner-product norm

Not all Banach spaces are Hilbert spaces. A norm comes from an inner product only if it satisfies the parallelogram law [3]. The spaces ℓp\ell^p and LpL^p for p≠2p\neq 2 are complete normed spaces but their norms violate the parallelogram law — they are Banach but not Hilbert. The case p=2p=2 is special precisely because it carries an inner product.

Counterexampleℓ1\ell^1 is not a Hilbert space

Take x=e1=(1,0,0,… )x = e_1 = (1,0,0,\dots) and y=e2=(0,1,0,… )y = e_2 = (0,1,0,\dots) in ℓ1\ell^1 with ∥z∥1=∑∣zk∣\|z\|_1 = \sum|z_k|. Then ∥x∥1=∥y∥1=1\|x\|_1 = \|y\|_1 = 1, and ∥x+y∥1=2\|x+y\|_1 = 2, ∥x−y∥1=2\|x-y\|_1 = 2. The parallelogram law would require

9
∥x+y∥2+∥x−y∥2=4+4=82∥x∥2+2∥y∥2=2+2=4.\quad \|x+y\|^2 + \|x-y\|^2 = 4+4 = 8 \overset{?}{=} 2\|x\|^2+2\|y\|^2 = 2+2 = 4.

Since 8≠48 \neq 4, no inner product induces the ℓ1\ell^1 norm.

Derivation hint: just compute the four ℓ1\ell^1 norms of e1±e2e_1\pm e_2 and substitute into [3].

⚠️ Closedness is essential in the Projection Theorem

The decomposition H=M⊕M⊥H = M \oplus M^\perp requires MM to be closed. For a non-closed subspace the infimum distance need not be attained, and M⊕M⊥M\oplus M^\perp may fail to be all of HH. Example: the space of finitely-supported sequences in ℓ2\ell^2 is dense but not closed; its orthogonal complement is {0}\{0\}, yet it is not all of ℓ2\ell^2, so M⊕M⊥≠HM\oplus M^\perp \neq H.

CounterexampleInfinite-dimensional unit ball is not compact

In finite dimensions the closed unit ball is compact (Heine–Borel). In any infinite-dimensional Hilbert space it is not: the orthonormal sequence (en)(e_n) has ∥en∥=1\|e_n\|=1 for all nn, yet ∥en−em∥2=2\|e_n - e_m\|^2 = 2 for n≠mn\neq m, so no subsequence is Cauchy and none converges. The geometry is genuinely "bigger" than intuition from R3\mathbb{R}^3.

Connections

connection

Hilbert spaces sit at the crossroads of several theories. They are the special Banach spaces (Banach Spaces) whose norm satisfies the parallelogram law, so all of functional-analytic machinery applies, plus extra geometry. The Spectral Theorem for self-adjoint and normal operators is the infinite-dimensional generalization of diagonalizing symmetric matrices, and it lives most naturally on Hilbert spaces. Fourier Series and the Fourier Transform are concrete instances of expanding a vector in an orthonormal basis of L2L^2. The Riesz Representation Theorem makes HH self-dual (H≅H∗H \cong H^*), a property failing for general Banach spaces. Finally, the abstract definition is the mathematical backbone of Quantum Mechanics, where states are rays in a complex Hilbert space.

Real-World Applications

🔬 Where this appears

Historical Development

⏳ History
- 1888–1900 — Inner products in mechanics/geometry. Quadratic forms and Euclidean dot products are well understood; Gram and Schmidt formalize orthogonalization.
- 1904–1910 — Hilbert's work on integral equations. David Hilbert studies infinite systems of equations and quadratic forms in infinitely many variables, implicitly using ℓ2\ell^2.
- 1907 — Riesz–Fischer Theorem. Frigyes Riesz and Ernst Fischer prove L2L^2 and ℓ2\ell^2 are isometric via Fourier coefficients — the cornerstone linking function spaces to sequence spaces.
- 1907 — Riesz Representation Theorem. Riesz characterizes continuous linear functionals on L2L^2, establishing self-duality.
- 1918 — Banach spaces emerge. Stefan Banach and others develop the general theory of complete normed spaces, of which Hilbert spaces are the geometrically richest example.
- 1927–1929 — Schmidt and the geometric language. Erhard Schmidt formulates the geometry of ℓ2\ell^2 (orthogonality, projection) in vector terms.
- 1929–1932 — von Neumann's axioms. John von Neumann gives the modern abstract definition of a (separable) Hilbert space and applies it in Mathematical Foundations of Quantum Mechanics (1932), unifying matrix and wave mechanics.
- 1930s — Spectral theory matures. von Neumann and M. H. Stone establish the spectral theorem for unbounded self-adjoint operators, essential for quantum dynamics.
- 1950 — RKHS theory. Nachman Aronszajn systematizes reproducing kernel Hilbert spaces, later central to statistics and machine learning.

Subtleties & Limits

open-problem · Non-separability, bases, and structural questions

Most applications use separable Hilbert spaces (those with a countable orthonormal basis), all isomorphic to ℓ2\ell^2. Non-separable Hilbert spaces (e.g. almost-periodic functions, certain spaces in quantum field theory) require an uncountable orthonormal basis and a transfinite version of Bessel/Parseval; their analysis is subtler and their use in physics (e.g. the "fermionic Fock space" subtleties, or representations of canonical commutation relations) remains an active interface. A deeper structural issue: while finite-dimensional inner product spaces are completely classified by dimension, and separable infinite-dimensional ones are all ≅ℓ2\cong \ell^2, the operator theory on them is vastly richer — the invariant subspace problem (does every bounded operator on a separable Hilbert space have a nontrivial closed invariant subspace?) is still open for Hilbert spaces, despite being resolved (negatively) on some Banach spaces. This shows the geometry is clean but the dynamics of operators are not.

See Also

Banach Spaces · Riesz Representation Theorem · Riesz–Fischer Theorem · Spectral Theorem · Orthonormal Basis · Fourier Series · Bounded Linear Operators · Quantum Mechanics