Convolution Theorem and Applications

⚡ Key Takeaways

One-line statement: The Fourier transform of a convolution of two functions equals the (pointwise) product of their individual Fourier transforms, and vice versa.

Key result in words: Convolution in one domain (time/space) becomes multiplication in the other domain (frequency). This turns hard integral operations into simple multiplications.

Where it applies: Signal processing, linear systems, probability (sums of random variables), optics, image processing, differential equations, acoustics.

Who/when: Rooted in the work of Joseph Fourier (1807–1822); the convolution operation and its transform property were formalized through the 19th and 20th centuries, becoming central to engineering with the Fast Fourier Transform (Cooley–Tukey, 1965).

💡 The Core Insight

A linear system is completely characterized by how it responds to a single impulse. Any input can be decomposed into a continuum of scaled, shifted impulses, and by linearity and time-invariance, the output is the superposition of shifted impulse responses — this superposition is convolution. The Fourier transform diagonalizes this operation because complex exponentials $e^{i\omega t}$ are the eigenfunctions of every LTI system: feeding in a pure frequency yields the same frequency scaled by a complex number. Convolution, which mixes everything in the time domain, becomes independent scaling of each frequency — i.e., pointwise multiplication.

DefinitionConvolution

The convolution of two functions $f, g: \mathbb{R} \to \mathbb{C}$ is defined by

1

\quad (f * g)(t) = \int_{-\infty}^{\infty} f(\tau)\, g(t - \tau)\, d\tau

Derivation hint: For each output time $t$ , flip $g$ about the origin, shift it by $t$ , multiply by $f$ , and integrate over all $\tau$ . The variable $\tau$ slides $g$ across $f$ . Here $t$ is the output (time or space) variable, $\tau$ is the integration (dummy) variable, $f$ and $g$ are the two functions being combined, and the result $(f*g)$ is itself a function of $t$ . Convolution is commutative: $f*g = g*f$ .

DefinitionFourier Transform Convention

We adopt the angular-frequency convention:

2

\quad \hat{f}(\omega) = \mathcal{F}\{f\}(\omega) = \int_{-\infty}^{\infty} f(t)\, e^{-i\omega t}\, dt

Derivation hint: Project $f$ onto the basis of complex exponentials; $\hat f(\omega)$ is the "amount" of frequency $\omega$ present in $f$ . where $\omega$ is angular frequency (rad/s), $i = \sqrt{-1}$ , and $\hat f$ is the transform. The inverse is $f(t) = \frac{1}{2\pi}\int \hat f(\omega) e^{i\omega t} d\omega$ .

TheoremConvolution Theorem

Let $f, g \in L^1(\mathbb{R})$ . Then their convolution is also integrable, and

3

\quad \mathcal{F}\{f * g\}(\omega) = \hat{f}(\omega)\,\hat{g}(\omega)

Derivation hint: Substitute the definition of convolution into the transform integral and exchange the order of integration; the shift produces a factored exponential. The dual (modulation) form states that pointwise multiplication in time corresponds to convolution in frequency (up to a $2\pi$ factor):

4

\quad \mathcal{F}\{f \cdot g\}(\omega) = \frac{1}{2\pi}\,(\hat{f} * \hat{g})(\omega)

Derivation hint: Apply the theorem in reverse using the inverse transform, where the $1/2\pi$ from the inverse convention carries through.

ProofProof of the Convolution Theorem

We compute the Fourier transform of $(f*g)$ directly.

Step 1. Write the transform of the convolution:

5

\quad \mathcal{F}\{f*g\}(\omega) = \int_{-\infty}^{\infty}\left[\int_{-\infty}^{\infty} f(\tau)\,g(t-\tau)\,d\tau\right] e^{-i\omega t}\, dt

Derivation hint: Insert equation [1] into equation [2].

Step 2. Because $f, g \in L^1$ , the double integral converges absolutely, so by Fubini's theorem we swap the order of integration:

6

\quad = \int_{-\infty}^{\infty} f(\tau) \left[\int_{-\infty}^{\infty} g(t-\tau)\, e^{-i\omega t}\, dt\right] d\tau

Derivation hint: Pull $f(\tau)$ out of the inner integral since it does not depend on $t$ .

Step 3. Substitute $u = t - \tau$ (so $t = u + \tau$ , $dt = du$ ) in the inner integral:

7

\quad \int_{-\infty}^{\infty} g(u)\, e^{-i\omega(u+\tau)}\, du = e^{-i\omega\tau} \int_{-\infty}^{\infty} g(u)\, e^{-i\omega u}\, du = e^{-i\omega\tau}\,\hat{g}(\omega)

Derivation hint: The exponential splits: $e^{-i\omega(u+\tau)} = e^{-i\omega\tau} e^{-i\omega u}$ ; the $\tau$ factor is constant w.r.t. $u$ .

Step 4. Substitute back:

8

\quad \mathcal{F}\{f*g\}(\omega) = \int_{-\infty}^{\infty} f(\tau)\, e^{-i\omega\tau}\,\hat{g}(\omega)\, d\tau = \hat{g}(\omega)\int_{-\infty}^{\infty} f(\tau)\,e^{-i\omega\tau}\,d\tau = \hat{f}(\omega)\,\hat{g}(\omega)

Derivation hint: $\hat g(\omega)$ is constant in $\tau$ and factors out, leaving exactly the transform of $f$ . $\blacksquare$

∎

mathematician · Joseph Fourier | 1768–1830

Jean-Baptiste Joseph Fourier introduced, in his 1822 Théorie analytique de la chaleur, the radical claim that any function could be represented as a sum (or integral) of sines and cosines. While studying heat diffusion, he developed the transform machinery from which the convolution theorem flows. His ideas were initially resisted by Lagrange and Laplace over questions of convergence, but they became one of the most powerful tools in all of applied mathematics. The decomposition of signals into frequencies — and the consequent simplification of convolution — is his enduring legacy.

🧠 Intuition

Think of convolution as weighted blurring. If $g$ is a narrow bump (a "kernel") and $f$ is a signal, then $(f*g)$ replaces each point of $f$ with a $g$ -shaped average of its neighbors. A wide $g$ blurs heavily; a delta-spike $g$ leaves $f$ unchanged ( $f * \delta = f$ ). Now imagine each frequency component of $f$ as an independent "channel." An LTI system can only scale and phase-shift each channel — it cannot create new frequencies. The number it multiplies each channel by is precisely $\hat g(\omega)$ , the frequency response. So blurring in time = attenuating high frequencies, which is exactly multiplication by a $\hat g$ that decays at high $\omega$ . The theorem says: the messy overlap-and-integrate dance in time is just dial-turning in frequency.

Figure

Convolution as flip-shift-multiply-integrate: the kernel

g

slides across

f

, and the shaded overlap area produces one output value.

ExampleTwo Rectangular Pulses → Triangle

Given: Let $f(t) = g(t) = \operatorname{rect}(t)$ , the unit rectangular pulse equal to $1$ for $-\tfrac12 \le t \le \tfrac12$ and $0$ otherwise.

Find: $(f*g)(t)$ , verified two ways.

Direct (time domain): The convolution of two unit-width rectangles is a triangle of base width $2$ and peak height $1$ :

9

\quad (f*g)(t) = \operatorname{tri}(t) = \begin{cases} 1 - |t|, & |t| \le 1 \\ 0, & |t| > 1\end{cases}

Derivation hint: The overlap of two unit rectangles, one shifted by $t$ , has length $1 - |t|$ for $|t|<1$ .

Frequency check: The transform of $\operatorname{rect}$ is $\hat f(\omega) = \operatorname{sinc}(\omega/2) = \dfrac{\sin(\omega/2)}{\omega/2}$ . By the theorem the transform of the convolution is

10

\quad \widehat{(f*g)}(\omega) = \hat f(\omega)\,\hat g(\omega) = \left(\frac{\sin(\omega/2)}{\omega/2}\right)^2 = \operatorname{sinc}^2(\omega/2)

Derivation hint: Square the sinc — and indeed $\operatorname{sinc}^2$ is the known transform of the triangle, confirming the result.

Boxed answer:

\boxed{(f*g)(t) = \operatorname{tri}(t),\qquad \widehat{(f*g)}(\omega) = \operatorname{sinc}^2(\omega/2)}

ExampleSum of Two Dice via Convolution of PMFs

Given: Two fair six-sided dice. Each die's outcome has probability mass function (PMF) $p(k) = \tfrac16$ for $k = 1,\dots,6$ . The PMF of the sum $S$ of two independent dice is the discrete convolution $p * p$ .

Compute: The discrete convolution

11

\quad (p*p)(n) = \sum_{k} p(k)\,p(n-k) = \frac{1}{36}\,(\text{number of ways two dice sum to } n)

Derivation hint: Independence makes the joint PMF a product; summing over all $(k, n-k)$ pairs counts the favorable outcomes. Evaluating for $n = 2,\dots,12$ gives counts $1,2,3,4,5,6,5,4,3,2,1$ , hence:

Sum $n$	2	3	4	5	6	7	8	9	10	11	12
$P(S=n)$	1/36	2/36	3/36	4/36	5/36	6/36	5/36	4/36	3/36	2/36	1/36

This is the discrete triangular distribution — the same triangle shape as Example 1, reflecting that adding independent variables convolves their distributions.

Boxed answer:

\boxed{P(S=7) = \frac{6}{36} = \frac{1}{6} \approx 0.167\ \text{(the most probable sum)}}

Probabilistic note: Taking the characteristic function (the Fourier transform of the PMF) turns this convolution into the product $\varphi_S(t) = \varphi(t)^2$ — the engine behind much of probability theory.

⚠️ Convolution is not multiplication in the same domain

A frequent error is to compute $(f*g)(t)$ as $f(t)g(t)$ . These are entirely different operations. Multiplication in the time domain corresponds to convolution in the frequency domain (equation [4]), not the reverse. Confusing the two leads, for instance, to wrong filter outputs. Also note the $2\pi$ factor in the dual form — it depends on your transform convention. With the symmetric ( $1/\sqrt{2\pi}$ ) convention or the ordinary-frequency convention ( $e^{-2\pi i f t}$ ), the constants move around. Always state your convention before quoting the theorem.

CounterexampleThe theorem can fail without integrability/decay assumptions

The theorem in form [3] requires $f, g \in L^1$ so the convolution exists and Fubini applies. Consider $f(t) = g(t) = 1$ (constant functions). Their convolution $\int_{-\infty}^{\infty} 1\cdot 1\, d\tau$ diverges — it does not exist as an ordinary function. Meanwhile their Fourier transforms are $\hat f = \hat g = 2\pi\,\delta(\omega)$ , and the naïve product $(2\pi)^2 \delta(\omega)^2$ is undefined (the square of a Dirac delta has no meaning). The clean identity holds only when both sides are well-defined; extending it to such cases requires the theory of tempered distributions and care about which products are admissible.

connection

The convolution theorem is the hub linking several areas. In Linear Time-Invariant Systems, the output is $y = x * h$ where $h$ is the impulse response, so $\hat y = \hat x \,\hat h$ — defining the transfer function $\hat h(\omega)$ . In Laplace Transform, the analogous property $\mathcal{L}\{f*g\} = F(s)G(s)$ governs circuit and control analysis. In Probability Theory, the density of a sum of independent random variables is the convolution of their densities, transformed via the characteristic function. In Z-Transform, discrete convolution of sequences maps to multiplication, foundational to digital filters. The theorem also underlies the Cross-Correlation and autocorrelation operations through the Wiener–Khinchin theorem, which ties autocorrelation to the power spectral density.

🔬 Concrete Uses

⏳ History

- 1754 — d'Alembert uses an integral resembling convolution in a Taylor-series remainder, an early appearance of the operation.

- 1807 — Fourier submits his memoir on heat to the French Academy, proposing trigonometric-series decompositions; the seed of frequency-domain analysis.

- 1822 — Fourier publishes Théorie analytique de la chaleur, establishing the Fourier integral and transform.

- 1880s — Wiener, Heaviside (operational calculus) develop operator methods that implicitly use convolution-as-multiplication for solving differential equations in engineering.

- 1934 — Norbert Wiener formalizes generalized harmonic analysis; the Wiener–Khinchin theorem links autocorrelation and spectrum.

- 1940s — Claude Shannon & engineers apply convolution rigorously to communication and sampling theory; LTI-system theory matures.

- 1947 — Laurent Schwartz founds the theory of distributions, giving rigorous meaning to convolutions of objects like the Dirac delta.

- 1965 — Cooley & Tukey publish the Fast Fourier Transform, reducing the DFT to

O(n\log n)

and making fast convolution practical — revolutionizing digital signal processing.

- 1980s–present — Convolution becomes the computational backbone of digital imaging, MP3/JPEG codecs, and (2010s) deep learning via CNNs.

open-problem · Boundaries of the Theorem

The clean identity assumes enough regularity ( $L^1$ , or appropriate distributional setting). Several genuine subtleties remain instructive at and beyond the undergraduate level:

Existence of $f*g$ . For $f \in L^p$ and $g \in L^q$ , Young's convolution inequality guarantees $f*g \in L^r$ with $\tfrac1r = \tfrac1p + \tfrac1q - 1$ ; the sharp constants in this inequality (Beckner, Brascamp–Lieb) connect to deep geometric-analytic questions.
Products of distributions. Multiplying two distributions (the dual side of convolution) is generally ill-defined; Hörmander's wavefront-set criterion specifies when it works — central to quantum field theory's renormalization difficulties.
Circular vs. linear convolution. The DFT computes circular convolution; recovering linear convolution requires zero-padding. Off-by-one and aliasing errors here are perennial practical pitfalls.
Deconvolution ill-posedness. Dividing by $\hat g(\omega)$ to invert a blur amplifies noise wherever $\hat g$ is small — an inverse problem requiring regularization (Tikhonov, Wiener filtering); robust blind deconvolution remains an active research area.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also