Functional analysis — Hilbert & Banach spaces

⚡ Key Takeaways

Functional analysis studies infinite-dimensional vector spaces equipped with a topology compatible with their linear structure, focusing on Banach spaces (complete normed spaces) and Hilbert spaces (complete inner product spaces). Key result: the geometry of Hilbert spaces (orthogonality, projection, self-duality) generalizes Euclidean geometry to infinite dimensions, while Banach space theory captures the broader world where norms need not come from inner products. Applies to PDE, quantum mechanics, signal processing, ergodic theory, and numerical analysis. Founded by Hilbert, Fréchet, Riesz, and Banach (1900s–1930s).

💡 The Core Insight

The single organizing principle is completeness married to linearity gives geometry and analysis simultaneously. In a Hilbert space the inner product encodes an exact notion of angle and orthogonal projection, making the space its own dual (Riesz Representation Theorem) and giving the best-approximation/projection theorem that drives least squares, Fourier series, and quantum measurement. In a general Banach space the inner product is gone, so duality becomes a genuinely separate object and the three pillars — Hahn–Banach, Open Mapping, Uniform Boundedness — replace orthogonality as the engines that make the theory run.

DefinitionNormed and Banach Spaces

A norm on a vector space $X$ over $\mathbb{K}\in\{\mathbb{R},\mathbb{C}\}$ is a map $\|\cdot\|:X\to[0,\infty)$ with:

1

\quad \|x\|=0 \iff x=0,\quad \|\lambda x\|=|\lambda|\,\|x\|,\quad \|x+y\|\le \|x\|+\|y\|

Derivation hint: These are positivity/definiteness, absolute homogeneity, and the triangle inequality; they make $d(x,y)=\|x-y\|$ a translation-invariant metric. The pair $(X,\|\cdot\|)$ is a Banach space if it is complete in the metric $d(x,y)=\|x-y\|$ : every Cauchy sequence converges in $X$ . Here $X$ is the space, $\lambda\in\mathbb{K}$ a scalar, and $x,y\in X$ vectors.

DefinitionInner Product and Hilbert Spaces

An inner product on $X$ over $\mathbb{C}$ is a map $\langle\cdot,\cdot\rangle:X\times X\to\mathbb{C}$ that is linear in the first argument, conjugate-symmetric, and positive-definite:

2

\quad \langle x,x\rangle\ge 0,\quad \langle x,x\rangle=0\iff x=0,\quad \langle y,x\rangle=\overline{\langle x,y\rangle}

Derivation hint: Conjugate symmetry forces $\langle x,x\rangle\in\mathbb{R}$ ; combined with sesquilinearity it determines the full form via the polarization identity. The induced norm is $\|x\|=\langle x,x\rangle^{1/2}$ . If $X$ is complete in this norm it is a Hilbert space $H$ . Two vectors are orthogonal, $x\perp y$ , when $\langle x,y\rangle=0$ .

TheoremCauchy–Schwarz and the Parallelogram Law

For all $x,y$ in an inner product space:

3

\quad |\langle x,y\rangle|\le \|x\|\,\|y\|

Derivation hint: Expand $0\le\langle x-\lambda y,\,x-\lambda y\rangle$ and minimize over $\lambda=\langle x,y\rangle/\|y\|^2$ .

4

\quad \|x+y\|^2+\|x-y\|^2 = 2\|x\|^2+2\|y\|^2

Derivation hint: Expand both squared norms via the inner product; cross terms cancel. Equation [4] characterizes Hilbert spaces: a Banach norm comes from an inner product iff it satisfies the parallelogram law (Jordan–von Neumann), with the inner product recovered by polarization

5

\quad \langle x,y\rangle = \tfrac14\sum_{k=0}^{3} i^{k}\,\|x+i^{k}y\|^2

Derivation hint: Substitute the four values $y\mapsto i^k y$ into $\|x+y\|^2=\|x\|^2+\|y\|^2+2\operatorname{Re}\langle x,y\rangle$ and combine.

TheoremHilbert Space Projection Theorem

Let $K\subseteq H$ be a nonempty closed convex set and $x\in H$ . Then there exists a unique $p\in K$ minimizing distance:

6

\quad \|x-p\| = \min_{k\in K}\|x-k\|

Derivation hint: Take a minimizing sequence $k_n$ ; the parallelogram law shows $\{k_n\}$ is Cauchy, completeness gives the limit, convexity keeps it in $K$ , and strict convexity gives uniqueness. When $K=M$ is a closed subspace, $p=P_M x$ is the orthogonal projection, characterized by $x-P_Mx\perp M$ , and $H=M\oplus M^\perp$ .

TheoremRiesz Representation Theorem

Every continuous linear functional $\varphi\in H^*$ is represented by a unique $y_\varphi\in H$ :

7

\quad \varphi(x)=\langle x,y_\varphi\rangle\quad\text{for all }x\in H,\qquad \|\varphi\|_{H^*}=\|y_\varphi\|_H

Derivation hint: If $\varphi=0$ take $y=0$ ; otherwise $\ker\varphi$ is closed with $1$ -dimensional orthogonal complement spanned by some $z$ , and $y_\varphi=\overline{\varphi(z)}\,z/\|z\|^2$ . Thus a Hilbert space is isometrically anti-isomorphic to its own dual, $H\cong H^*$ .

ProofProjection Theorem via the Parallelogram Law

Let $K$ be closed and convex, $d=\inf_{k\in K}\|x-k\|$ , and pick $k_n\in K$ with $\|x-k_n\|\to d$ . 1. Apply [4] to $u=x-k_n$ , $v=x-k_m$ :

8

\quad \|k_n-k_m\|^2 = 2\|x-k_n\|^2+2\|x-k_m\|^2-4\Big\|x-\tfrac{k_n+k_m}{2}\Big\|^2

Derivation hint: Set $u-v=k_m-k_n$ and $u+v=2x-(k_n+k_m)$ in the parallelogram identity. 2. By convexity $\tfrac{k_n+k_m}{2}\in K$ , so $\big\|x-\tfrac{k_n+k_m}{2}\big\|\ge d$ . Hence

9

\quad \|k_n-k_m\|^2 \le 2\|x-k_n\|^2+2\|x-k_m\|^2-4d^2 \;\xrightarrow[n,m\to\infty]{}\; 0

Derivation hint: Both $\|x-k_n\|^2,\|x-k_m\|^2\to d^2$ , so the right side $\to 4d^2-4d^2=0$ . 3. So $\{k_n\}$ is Cauchy; by completeness $k_n\to p\in H$ , and closedness gives $p\in K$ with $\|x-p\|=d$ . 4. Uniqueness: if $p,p'$ both attain $d$ , step 1 with $k_n=p,k_m=p'$ gives $\|p-p'\|^2\le 4d^2-4d^2=0$ . 5. Orthogonality (subspace case): for $m\in M$ and $t\in\mathbb{R}$ , $\|x-p-tm\|^2\ge\|x-p\|^2$ expands to $-2t\operatorname{Re}\langle x-p,m\rangle+t^2\|m\|^2\ge 0$ for all $t$ , forcing $\operatorname{Re}\langle x-p,m\rangle=0$ ; replacing $m$ by $im$ kills the imaginary part. $\blacksquare$

∎

ProofThe Three Pillars of Banach Space Theory

These hold in every Banach space and need no inner product. 1. Hahn–Banach: a bounded functional on a subspace $Y\subseteq X$ extends to all of $X$ with the same norm:

10

\quad \exists\,\Phi\in X^*:\ \Phi|_Y=\varphi,\quad \|\Phi\|_{X^*}=\|\varphi\|_{Y^*}

Derivation hint: Extend one dimension at a time controlling by a sublinear functional, then invoke Zorn's lemma for a maximal extension. 2. Uniform Boundedness (Banach–Steinhaus): a pointwise-bounded family $\{T_\alpha\}$ of bounded operators on a Banach space is uniformly bounded:

11

\quad \sup_\alpha\|T_\alpha x\|<\infty\ \forall x \ \Longrightarrow\ \sup_\alpha\|T_\alpha\|<\infty

Derivation hint: Baire category — one of the closed sets $\{x:\sup_\alpha\|T_\alpha x\|\le n\}$ has nonempty interior. 3. Open Mapping / Closed Graph: a bounded surjection between Banach spaces is open; equivalently a closed-graph linear map is bounded.

12

\quad T\in\mathcal B(X,Y)\text{ surjective}\ \Longrightarrow\ T \text{ open};\qquad \Gamma(T)\text{ closed}\Longrightarrow T\text{ bounded}

Derivation hint: Baire category gives $T(B_X)$ dense and "fat"; a completeness/iteration argument upgrades density to an open neighborhood of $0$ . $\blacksquare$

∎

mathematician · Stefan Banach | 1892–1945

Banach, largely self-taught and working in Lwów, axiomatized complete normed spaces in his 1922 thesis and his 1932 monograph Théorie des opérations linéaires, which gave the field its name and its central theorems (open mapping, closed graph, Hahn–Banach in its analytic form). He led the legendary Scottish Café seminar, where problems were recorded in the Scottish Book. His insistence on abstract axioms over concrete function spaces — earlier developed by Fréchet, F. Riesz, and Hilbert — turned a collection of techniques into a coherent geometric theory.

🧠 Intuition

Think of a Hilbert space as $\mathbb{R}^n$ with $n=\infty$ but the geometry intact: you can still drop a perpendicular, take an orthonormal basis, expand any vector in coordinates (Fourier coefficients), and Pythagoras holds. The closed-subspace projection is literally "the foot of the perpendicular," which is why least-squares regression, conditional expectation in probability, and quantum measurement are all the same theorem. A general Banach space is what you get when you keep distances but throw away angles. You can still measure lengths and converge, but there is no canonical perpendicular, so "the closest point in a subspace" may not be unique and the space need not equal $M\oplus M^\perp$ . This is why $\ell^p$ and $L^p$ for $p\neq2$ feel "bent": their unit balls have corners or flat faces rather than being round, and that lost roundness is exactly the failed parallelogram law [4].

Figure

Unit balls: roundness of

\ell^2

vs. corners/flatness of

\ell^1

and

\ell^\infty

, illustrating why only

p=2

is a Hilbert space

ExampleBest Approximation by Fourier Coefficients in

L^2

Given: $H=L^2([-\pi,\pi])$ with $\langle f,g\rangle=\int_{-\pi}^{\pi}f\bar g\,dx$ , orthonormal system $e_n(x)=\frac{1}{\sqrt{2\pi}}e^{inx}$ , and target $f(x)=x$ . Approximate $f$ in the subspace $M=\operatorname{span}\{e_{-1},e_0,e_1\}$ .

Steps: 1. The projection is $P_Mf=\sum_{n=-1}^{1}\langle f,e_n\rangle e_n$ (orthonormal coordinates). 2. Compute coefficients. $\langle x,e_0\rangle=\frac{1}{\sqrt{2\pi}}\int_{-\pi}^{\pi}x\,dx=0$ . 3. For $n=\pm1$ : $\langle x,e_n\rangle=\frac{1}{\sqrt{2\pi}}\int_{-\pi}^{\pi}x\,e^{-inx}dx$ . Integrating by parts, $\int_{-\pi}^{\pi}x e^{-inx}dx = \frac{2\pi}{n}(-1)^{n+1}\,i$ ... giving $\langle x,e_1\rangle = i\sqrt{2\pi}\cdot(-1)^{2}/1 = i\sqrt{2\pi}$ up to sign, and $\langle x,e_{-1}\rangle=-i\sqrt{2\pi}$ . 4. Recombining the $\pm1$ terms yields the real trigonometric form $P_Mf(x)=2\sin x$ (the first Fourier sine term of $x$ ). 5. The residual $f-P_Mf$ is orthogonal to $M$ by [6].

Answer: $\boxed{P_Mf(x)=2\sin x}$ — the unique closest function in $M$ to $f(x)=x$ in $L^2$ -norm.

ExampleOperator Norm and Spectrum of a Multiplication Operator

Given: On $H=L^2([0,1])$ define $(Tf)(x)=x\,f(x)$ .

Steps: 1. Boundedness: $\|Tf\|^2=\int_0^1 x^2|f|^2dx\le \int_0^1|f|^2dx=\|f\|^2$ , so $\|T\|\le1$ . 2. Sharpness: take $f_n$ concentrated near $x=1$ ; then $\|Tf_n\|/\|f_n\|\to1$ , so $\|T\|=1$ . 3. Self-adjointness: $\langle Tf,g\rangle=\int_0^1 x f\bar g=\langle f,Tg\rangle$ , hence $T=T^*$ . 4. Eigenvalues: $Tf=\lambda f$ means $(x-\lambda)f=0$ a.e., forcing $f=0$ — no eigenvalues. 5. Spectrum: $T-\lambda I$ is invertible iff $\frac{1}{x-\lambda}$ is bounded on $[0,1]$ , i.e. iff $\lambda\notin[0,1]$ . So $\sigma(T)=[0,1]$ .

Answer: $\boxed{\|T\|=1,\quad \sigma(T)=[0,1],\quad \text{point spectrum }=\varnothing}$ — a self-adjoint operator with purely continuous spectrum, impossible in finite dimensions.

⚠️ Common mistake

"Bounded and closed implies compact." False in infinite dimensions. By Riesz's lemma, the closed unit ball of a normed space is compact iff the space is finite-dimensional. This is why we need weak/weak-* topologies (Banach–Alaoglu) to recover any compactness, and why "compact operator" is a special, restrictive class rather than the norm.

⚠️ Common mistake

"Every closed subspace has an orthogonal complement / every functional is an inner product." True in Hilbert spaces, false in general Banach spaces. The Riesz representation [7] and the decomposition $H=M\oplus M^\perp$ are Hilbert-only luxuries. In $\ell^1$ or $C[0,1]$ , not every closed subspace is complemented, and the dual is a genuinely different space ( $(\ell^1)^*=\ell^\infty$ , $(c_0)^*=\ell^1$ ).

CounterexampleA Discontinuous Linear Functional and Non-Convergent Fourier Series

On the incomplete space of finitely-supported sequences with $\ell^2$ -norm, the functional $\varphi(x)=\sum_n n\,x_n$ is linear but unbounded ( $\varphi(e_n)=n$ while $\|e_n\|=1$ ). Such pathologies require the axiom of choice to exhibit on complete spaces — consistent with the existence of a Hamel basis. Separately, Du Bois-Reymond's example (1873) gives a continuous function whose Fourier series diverges at a point: pointwise convergence fails even for continuous data. This is invisible in $L^2$ , where Fourier series always converge in norm by Orthonormal Bases & Bessel's Inequality — a vivid demonstration that the choice of norm changes which theorems are true.

connection

Functional analysis sits at the crossroads of several theories. The spectral theorem for self-adjoint and compact operators (Spectral Theory of Operators) generalizes the diagonalization of symmetric matrices and underlies Quantum Mechanics Mathematical Foundations, where observables are self-adjoint operators and states are unit vectors. The duality theory connects to Locally Convex & Topological Vector Spaces (where weak topologies and Hahn–Banach separation live) and to Distributions & Sobolev Spaces, the completed function spaces in which modern PDE theory operates. The projection theorem [6] is the abstract heart of Conditional Expectation & Martingales in probability and of Least Squares & Variational Methods. Finally, Fourier Analysis & Harmonic Analysis is essentially the spectral theory of translation-invariant operators on $L^2$ .

🔬 Where this appears

⏳ History

- 1900–1906: Hilbert and Schmidt study integral equations and infinite quadratic forms, introducing

\ell^2

"Hilbert space" implicitly via spectral ideas.

- 1906: Fréchet's thesis introduces abstract metric spaces, the framework for completeness.

- 1907: F. Riesz and Fischer prove the Riesz–Fischer theorem (

L^2

completeness and Fourier-coefficient correspondence), and Riesz proves the representation theorem for

L^2

.

- 1910–1918: Riesz develops

L^p

and

\ell^p

spaces and bounded operators; F. Riesz proves the lemma on non-compactness of the unit ball.

- 1922: Banach's doctoral thesis axiomatizes complete normed spaces ("spaces of type (B)"); Wiener independently formulates similar ideas.

- 1927: Hahn, and independently Banach (1929), prove the analytic Hahn–Banach extension theorem.

- 1929–1932: von Neumann gives the axiomatic definition of (abstract, separable) Hilbert space and the spectral theorem for unbounded self-adjoint operators, motivated by quantum mechanics.

- 1932: Banach's Théorie des opérations linéaires unifies the field; Banach–Steinhaus and the open mapping theorem appear.

- 1934–1940s: Alaoglu (weak-* compactness, 1940), Gelfand (Banach algebras, late 1930s), and Eidelheit/Mazur (separation theorems) round out the structural theory.

open-problem

Several deep structural questions remain. The invariant subspace problem — does every bounded operator on a separable infinite-dimensional Hilbert space have a nontrivial closed invariant subspace? — is still open for Hilbert spaces, though Enflo (1987) and Read constructed Banach-space counterexamples. Enflo (1973) also resolved the approximation problem negatively (a Banach space without the approximation property), refuting a long-standing conjecture. The complemented subspace problem (only Hilbert spaces have all closed subspaces complemented) was settled by Lindenstrauss–Tzafriri (1971). Many isometric/isomorphic classification questions for $L^p$ and $C(K)$ spaces remain delicate, and the geometry of Banach spaces (type, cotype, finite representability) is an active research area linking probability, combinatorics, and operator theory.

Functional analysis — Hilbert & Banach spaces

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also