Approximation

⚡ Key Takeaways

Approximation is the deliberate replacement of an exact quantity by a nearby, simpler value that is good enough for a stated purpose. Key result: every approximation carries a controllable error, and good methods make that error as small as required. Where it applies: arithmetic, algebra, calculus, physics, engineering, computing — anywhere exact values are impossible, unknown, or unnecessary. Who/when: as old as counting; formalised by Archimedes (c. 250 BCE), and made rigorous by Cauchy, Weierstrass, and Taylor in the 18th–19th centuries.

💡 The Core Insight

Every useful approximation comes as a pair: a simpler value and a bound on how far it can be from the truth. A number without an error estimate is a guess; a number with one is a tool. The whole art is trading simplicity against accuracy, then proving the trade was safe.

DefinitionApproximation and Error

Let $x$ be an exact (target) value and $\tilde{x}$ an approximation to it. The absolute error is

1

\quad E_{\text{abs}} = |x - \tilde{x}|.

Derivation hint: take the magnitude of the signed difference $x - \tilde{x}$ so over- and under-estimates count equally. The relative error (when $x \neq 0$ ) is

2

\quad E_{\text{rel}} = \frac{|x - \tilde{x}|}{|x|},

Derivation hint: divide the absolute error by the size of the true value to make the error dimensionless and scale-independent. The percentage error is simply $100\% \times E_{\text{rel}}$ . Symbols: $x$ = true value, $\tilde{x}$ = approximate value, $|\cdot|$ = absolute value. We say $\tilde{x}$ is correct to $n$ significant figures if $E_{\text{rel}} < \tfrac{1}{2}\times 10^{-n+1}$ , and correct to $n$ decimal places if $E_{\text{abs}} < \tfrac{1}{2}\times 10^{-n}$ .

DefinitionOrder of Approximation

A family of approximations $\tilde{x}(h)$ depending on a small parameter $h$ has error of order $p$ , written $O(h^p)$ , if there is a constant $C$ with

3

\quad |x - \tilde{x}(h)| \le C\,|h|^{p} \quad \text{for all small } h.

Derivation hint: bound the error by a constant multiple of a power of the step size; the exponent $p$ is the "order." Larger $p$ means the error shrinks faster as $h \to 0$ : halving $h$ divides an $O(h^2)$ error by $4$ .

TheoremTaylor Approximation with Remainder

If $f$ is $(n+1)$ times differentiable near $a$ , then for $x$ near $a$ ,

4

\quad f(x) = \sum_{k=0}^{n} \frac{f^{(k)}(a)}{k!}(x-a)^k + R_n(x),

Derivation hint: match the polynomial's first $n$ derivatives at $a$ to those of $f$ , leaving a remainder. with the Lagrange remainder

5

\quad R_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!}(x-a)^{n+1}

for some $\xi$ between $a$ and $x$ . Derivation hint: apply the Mean Value Theorem repeatedly (or Cauchy's MVT) to the difference between $f$ and its polynomial. The polynomial part is a systematic approximation whose error $R_n$ can be bounded explicitly.

ProofLinear (Tangent-Line) Approximation

We derive the simplest non-trivial approximation, then bound its error. 1. Setup. Let $f$ be differentiable at $a$ . Near $a$ , replace the curve by its tangent line:

6

\quad L(x) = f(a) + f'(a)(x-a).

Derivation hint: the tangent line passes through $(a, f(a))$ with slope $f'(a)$ — point-slope form. 2. Why it's good. By the definition of derivative, $f(x) - f(a) = f'(a)(x-a) + o(x-a)$ , so $f(x) - L(x)$ vanishes faster than $(x-a)$ as $x \to a$ . 3. Quantify the error. If $f''$ exists and $|f''| \le M$ on the interval, Taylor's theorem with $n=1$ gives

7

\quad |f(x) - L(x)| = \left|\frac{f''(\xi)}{2}(x-a)^2\right| \le \frac{M}{2}(x-a)^2.

Derivation hint: substitute $n=1$ into the Lagrange remainder [5]. 4. Conclusion. The tangent-line error is $O\big((x-a)^2\big)$ : a second-order small quantity. Halving the distance from $a$ quarters the error. $\blacksquare$

∎

mathematician · Brook Taylor | 1685–1731

The English mathematician Brook Taylor published the series now bearing his name in Methodus Incrementorum Directa et Inversa (1715). Though Gregory and Newton had used special cases earlier, Taylor gave the general formula expressing a function as a polynomial built from its derivatives at a point. The crucial remainder term — which turns a formal series into a controlled approximation — came later from Lagrange and Cauchy. Taylor's work underlies essentially every practical method for approximating functions on computers today.

🧠 Intuition

Think of approximation as zooming in. A circle looks curved from far away, but zoom into a tiny arc and it looks straight — that "straightness" is the tangent-line approximation. The more you zoom (smaller $|x-a|$ ), the better a straight line fits, and the leftover bend shrinks quadratically. Higher-order approximations (parabolas, cubics) are like keeping more of the curve's "personality" — its curvature, its twist — so the fit stays good over a wider window. The error term is the universe's receipt: it tells you exactly what you threw away.

Figure

Tangent-line approximation: the curve

f(x)

versus its linear approximation

L(x)

at

a

. The gap is the error

R_1

.

```svg

```

ExampleApproximating

\sqrt{50}

by linearisation

Given: Estimate $\sqrt{50}$ without a calculator, using $f(x)=\sqrt{x}$ near $a=49$ (a perfect square).

Steps: 1. $f(a) = \sqrt{49} = 7$ . 2. $f'(x) = \tfrac{1}{2\sqrt{x}}$ , so $f'(49) = \tfrac{1}{14}$ . 3. Linear approximation [6]: $L(50) = 7 + \tfrac{1}{14}(50-49) = 7 + \tfrac{1}{14} \approx 7.0714$ . 4. Error bound via [7]: $f''(x) = -\tfrac{1}{4}x^{-3/2}$ , and $|f''| \le \tfrac{1}{4}(49)^{-3/2} = \tfrac{1}{1372}$ on $[49,50]$ . So error $\le \tfrac{1}{2}\cdot\tfrac{1}{1372}\cdot 1^2 \approx 0.00036$ .

Answer: $\boxed{\sqrt{50} \approx 7.0714,\ \text{error} < 0.0004}$ (true value $7.07107\ldots$ ). ✓

ExampleRelative error of a rounded measurement

Given: A length is measured as $\tilde{x} = 2.5\,\text{m}$ , rounded to the nearest $0.1\,\text{m}$ . Find the worst-case absolute and relative error.

Steps: 1. Rounding to nearest $0.1$ means the true value lies in $[2.45, 2.55]\,\text{m}$ . 2. Worst-case absolute error [1]: $E_{\text{abs}} = 0.05\,\text{m}$ . 3. Relative error [2]: $E_{\text{rel}} = \dfrac{0.05}{2.5} = 0.02 = 2\%$ .

Answer: $\boxed{E_{\text{abs}} = 0.05\,\text{m},\quad E_{\text{rel}} = 2\%}$ . ✓ Note the relative error has no units — it is a pure ratio.

⚠️ Common mistake

"More decimal places means more accurate." False. Writing $\pi \approx 3.141592653589793\ldots$ from a calculation that was only good to $3$ digits is false precision — the extra digits are noise. Accuracy is set by the method and the data, not by how many digits you copy down. Always round a final answer to the number of figures your inputs justify.

⚠️ Common mistake

"Small relative error always means small absolute error." Not when the quantity is huge. A $0.1\%$ error on a national budget of $4 trillion is $4 billion — small relatively, enormous absolutely. Choose the error measure that matches the decision you are making.

CounterexampleAn approximation that gets worse with more terms

One might assume that adding more terms of a series always improves the estimate. For an asymptotic series, this fails. For example, approximating $\int_0^\infty \frac{e^{-t}}{1+xt}\,dt$ by its asymptotic expansion $\sum (-1)^k k!\,x^k$ at small $x$ gives improving accuracy up to an optimal number of terms, after which the (divergent) terms blow up and the approximation degrades. More effort can make things worse — error control, not term-counting, is what matters.

connection

Approximation is the connective tissue of mathematics. The Derivative is the best linear approximation to a function. Taylor Series generalise this to polynomials of any order. Limits formalise the idea that an approximation can be driven to exactness. Newton's Method iterates linear approximations to solve equations. In statistics, Linear Regression is approximation of data by the closest line, and Numerical Integration approximates areas by sums. Even Significant Figures are a bookkeeping system for tracking how much approximation our numbers contain.

🔬 Where this appears

⏳ History

- c. 1800 BCE — Babylonian tablet YBC 7289 records

\sqrt{2} \approx 1.41421296

, accurate to six digits.

- c. 250 BCE — Archimedes bounds

\pi

between

\tfrac{223}{71}

and

\tfrac{22}{7}

using inscribed and circumscribed polygons — a rigorous interval approximation.

- c. 480 CE — Zu Chongzhi computes

\pi \approx 355/113

, correct to seven digits, unmatched for nearly a millennium.

- 1669 — Newton develops series methods and what becomes Newton's iteration for roots.

- 1715 — Brook Taylor publishes the general Taylor series.

- 1797 — Lagrange supplies a remainder term, turning the series into a bounded approximation.

- 1821 — Cauchy gives rigorous definitions of limit and error, founding analysis.

- 1885 — Weierstrass proves any continuous function on a closed interval can be uniformly approximated by polynomials (the Approximation Theorem).

- 1985 — IEEE 754 standardises floating-point approximation for all modern computers.

open-problem

Approximation theory still holds deep open questions. Optimal approximation: for a given function and a fixed "budget" (e.g. a polynomial of degree $n$ ), finding the provably best approximation in the supremum norm (the Chebyshev / minimax problem) is solved in principle but computationally hard in many real settings. High-dimensional approximation suffers the curse of dimensionality: the work to approximate a function of $d$ variables to fixed accuracy can grow exponentially in $d$ — a barrier central to modern machine learning, where why deep neural networks escape this curse for many real-world functions is not fully understood. Inverse problems — recovering an exact cause from approximate, noisy effects (as in medical imaging) — remain mathematically delicate and ill-posed.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also