Directional Derivatives and Differentiability

⚡ Key Takeaways

Directional derivatives measure the instantaneous rate of change of a multivariable function along a chosen direction; differentiability is the stronger condition that the function is well-approximated by a linear map at a point. Key result: differentiability implies all directional derivatives exist and equals the gradient dotted with the direction — but the converse is FALSE. Applies across multivariable calculus, optimization, physics, and machine learning. Crystallized in the 19th century through Cauchy, Weierstrass, and Fréchet.

💡 The Core Insight

Existence of directional derivatives is a one-dimensional condition checked along each line separately. Differentiability is a genuinely multidimensional condition: it demands that ONE linear map approximates $f$ uniformly as you approach $\mathbf{a}$ from all directions at once. The gap between these — sequential 1D limits versus a single 2D/nD limit — is the source of nearly every subtlety on this page.

DefinitionDirectional Derivative

Let $f:\mathbb{R}^n \to \mathbb{R}$ , let $\mathbf{a}\in\mathbb{R}^n$ , and let $\mathbf{u}\in\mathbb{R}^n$ be a unit vector ( $\|\mathbf{u}\|=1$ ). The directional derivative of $f$ at $\mathbf{a}$ in the direction $\mathbf{u}$ is

1

\quad D_{\mathbf{u}}f(\mathbf{a}) = \lim_{h\to 0}\frac{f(\mathbf{a}+h\mathbf{u}) - f(\mathbf{a})}{h}

Derivation hint: Restrict $f$ to the line $g(h)=f(\mathbf{a}+h\mathbf{u})$ , then $D_{\mathbf{u}}f(\mathbf{a})=g'(0)$ by the ordinary single-variable derivative definition. Here $\mathbf{a}=(a_1,\dots,a_n)$ is the base point, $\mathbf{u}=(u_1,\dots,u_n)$ the direction, and $h\in\mathbb{R}$ the scalar step. When $\mathbf{u}=\mathbf{e}_i$ (the $i$ -th standard basis vector), $D_{\mathbf{e}_i}f(\mathbf{a})$ is the partial derivative $\partial f/\partial x_i$ .

⚠️ Common mistake

Some authors do not require $\mathbf{u}$ to be a unit vector. If $\mathbf{u}$ is unnormalized, $D_{\mathbf{u}}f$ scales linearly with $\|\mathbf{u}\|$ , and the geometric interpretation as "rate per unit distance" is lost. Always check your textbook's convention. This page uses unit vectors.

DefinitionDifferentiability (Fréchet)

$f:\mathbb{R}^n\to\mathbb{R}$ is differentiable at $\mathbf{a}$ if there exists a linear map $L:\mathbb{R}^n\to\mathbb{R}$ (equivalently a vector $\nabla f(\mathbf{a})$ ) such that

2

\quad \lim_{\mathbf{h}\to\mathbf{0}}\frac{f(\mathbf{a}+\mathbf{h}) - f(\mathbf{a}) - L(\mathbf{h})}{\|\mathbf{h}\|} = 0

Derivation hint: This says the error of the linear approximation $f(\mathbf{a})+L(\mathbf{h})$ vanishes faster than $\|\mathbf{h}\|$ ; it is the multivariable analogue of $f(a+h)=f(a)+f'(a)h+o(h)$ . The linear map $L$ is the (total) derivative $Df(\mathbf{a})$ , and the vector representing it is the gradient $\nabla f(\mathbf{a})=\left(\frac{\partial f}{\partial x_1},\dots,\frac{\partial f}{\partial x_n}\right)$ , so that $L(\mathbf{h})=\nabla f(\mathbf{a})\cdot\mathbf{h}$ .

TheoremGradient Formula for Directional Derivatives

If $f$ is differentiable at $\mathbf{a}$ , then every directional derivative exists and is given by

3

\quad D_{\mathbf{u}}f(\mathbf{a}) = \nabla f(\mathbf{a})\cdot\mathbf{u} = \sum_{i=1}^{n}\frac{\partial f}{\partial x_i}(\mathbf{a})\,u_i

Derivation hint: Set $\mathbf{h}=h\mathbf{u}$ in equation [2], divide by $h$ , and take $h\to 0$ ; the $o(\|\mathbf{h}\|)$ term vanishes because $\|\mathbf{h}\|=|h|$ .

TheoremContinuity from Differentiability

If $f$ is differentiable at $\mathbf{a}$ , then $f$ is continuous at $\mathbf{a}$ .

4

\quad \lim_{\mathbf{h}\to\mathbf{0}} f(\mathbf{a}+\mathbf{h}) = f(\mathbf{a})

Derivation hint: From [2], $f(\mathbf{a}+\mathbf{h})-f(\mathbf{a}) = L(\mathbf{h}) + o(\|\mathbf{h}\|)$ ; both terms $\to 0$ as $\mathbf{h}\to\mathbf{0}$ since $L$ is linear hence continuous.

TheoremDirection of Steepest Ascent

Among all unit directions $\mathbf{u}$ , the directional derivative $D_{\mathbf{u}}f(\mathbf{a})$ is maximized when $\mathbf{u}$ points along $\nabla f(\mathbf{a})$ , with maximum value $\|\nabla f(\mathbf{a})\|$ .

5

\quad \max_{\|\mathbf{u}\|=1} D_{\mathbf{u}}f(\mathbf{a}) = \|\nabla f(\mathbf{a})\|, \quad \text{attained at } \mathbf{u}=\frac{\nabla f(\mathbf{a})}{\|\nabla f(\mathbf{a})\|}

Derivation hint: By [3] and Cauchy–Schwarz, $\nabla f\cdot\mathbf{u} = \|\nabla f\|\,\|\mathbf{u}\|\cos\theta = \|\nabla f\|\cos\theta$ , maximized at $\theta=0$ .

ProofDifferentiability ⟹ Gradient Formula (Equation 3)

We prove that differentiability at $\mathbf{a}$ forces $D_{\mathbf{u}}f(\mathbf{a})=\nabla f(\mathbf{a})\cdot\mathbf{u}$ for every unit $\mathbf{u}$ .

Step 1. By definition [2], write the remainder as

r(\mathbf{h}) = f(\mathbf{a}+\mathbf{h}) - f(\mathbf{a}) - \nabla f(\mathbf{a})\cdot\mathbf{h},

with the property $r(\mathbf{h})/\|\mathbf{h}\| \to 0$ as $\mathbf{h}\to\mathbf{0}$ .

Step 2. Specialize to $\mathbf{h}=h\mathbf{u}$ with $h\in\mathbb{R}$ , $h\neq 0$ . Since $\|\mathbf{u}\|=1$ , we have $\|\mathbf{h}\|=|h|$ . Then

f(\mathbf{a}+h\mathbf{u}) - f(\mathbf{a}) = \nabla f(\mathbf{a})\cdot(h\mathbf{u}) + r(h\mathbf{u}) = h\,\big(\nabla f(\mathbf{a})\cdot\mathbf{u}\big) + r(h\mathbf{u}).

Step 3. Divide by $h$ :

\frac{f(\mathbf{a}+h\mathbf{u}) - f(\mathbf{a})}{h} = \nabla f(\mathbf{a})\cdot\mathbf{u} + \frac{r(h\mathbf{u})}{h}.

Step 4. Estimate the remainder term. Since $|h|=\|h\mathbf{u}\|$ ,

\left|\frac{r(h\mathbf{u})}{h}\right| = \frac{|r(h\mathbf{u})|}{\|h\mathbf{u}\|} \xrightarrow{\;h\to 0\;} 0

by the defining property of $r$ in Step 1.

Step 5. Taking $h\to 0$ in Step 3 yields the limit $D_{\mathbf{u}}f(\mathbf{a}) = \nabla f(\mathbf{a})\cdot\mathbf{u}$ , which is equation [3]. $\blacksquare$

∎

TheoremSufficient Condition for Differentiability

If all partial derivatives $\partial f/\partial x_i$ exist in a neighborhood of $\mathbf{a}$ and are continuous at $\mathbf{a}$ , then $f$ is differentiable at $\mathbf{a}$ . (Such $f$ is called continuously differentiable, or $C^1$ .)

ProofSketch: Continuous Partials ⟹ Differentiable (n=2)

Step 1. Write the increment as a telescoping sum, changing one coordinate at a time:

f(a_1+h_1, a_2+h_2) - f(a_1,a_2) = \big[f(a_1+h_1,a_2+h_2)-f(a_1,a_2+h_2)\big] + \big[f(a_1,a_2+h_2)-f(a_1,a_2)\big].

Step 2. Apply the single-variable Mean Value Theorem to each bracket. There exist $\xi_1$ between $a_1$ and $a_1+h_1$ , and $\xi_2$ between $a_2$ and $a_2+h_2$ , with

= \frac{\partial f}{\partial x_1}(\xi_1, a_2+h_2)\,h_1 + \frac{\partial f}{\partial x_2}(a_1, \xi_2)\,h_2.

Step 3. Use continuity of the partials: as $\mathbf{h}\to\mathbf{0}$ , the evaluation points approach $\mathbf{a}$ , so

\frac{\partial f}{\partial x_i}(\text{point}) = \frac{\partial f}{\partial x_i}(\mathbf{a}) + \varepsilon_i(\mathbf{h}),\quad \varepsilon_i(\mathbf{h})\to 0.

Step 4. Substitute: the increment equals $\nabla f(\mathbf{a})\cdot\mathbf{h} + \varepsilon_1 h_1 + \varepsilon_2 h_2$ . The error term satisfies

\frac{|\varepsilon_1 h_1 + \varepsilon_2 h_2|}{\|\mathbf{h}\|} \le |\varepsilon_1| + |\varepsilon_2| \to 0,

verifying [2]. $\blacksquare$

∎

mathematician · Maurice Fréchet | 1878–1973

The French mathematician Maurice Fréchet gave the modern, rigorous definition of differentiability in normed and metric spaces (1911), generalizing the classical multivariable derivative to infinite-dimensional settings. The "Fréchet derivative" of equation [2] is now the standard definition. Fréchet also pioneered abstract metric spaces and functional analysis, providing the language in which "differentiability as approximation by a bounded linear operator" became precise. His framework distinguishes the strong (Fréchet) derivative from the weaker Gâteaux derivative, which is exactly the directional-derivative notion lifted to function spaces — making this very page's central tension a foundational theme of modern analysis.

🧠 Intuition

Imagine standing on a hillside whose height is $f(x,y)$ . A directional derivative is the slope you feel if you start walking in a fixed compass direction $\mathbf{u}$ — steepness measured along that one path. There are infinitely many such slopes, one per direction.

Differentiability is a much stronger claim: it says the hillside near your feet looks like a flat tilted plane (the tangent plane). If that's true, then every walking-direction's slope is just the projection of one master arrow — the gradient — onto your direction. The gradient points "straight uphill," and its length is the steepest possible slope. The crucial subtlety: a surface can have a well-defined slope along every straight-line path through a point and yet still be crinkled, creased, or torn there — so no single tangent plane fits. Directional derivatives only probe along straight lines; differentiability demands good behavior along every approach, including curved and oscillating ones. That's why all directional derivatives existing is not enough.

Figure

Gradient and directional derivatives on a level-set map. The gradient is perpendicular to level curves and gives the steepest slope; a directional derivative projects it onto a chosen direction.

```svg

```

ExampleExample 1 — Computing a Directional Derivative

Given: $f(x,y) = x^2 y + 3y^2$ at the point $\mathbf{a}=(1,2)$ , in the direction of $\mathbf{v}=(3,4)$ .

Step 1 — Normalize the direction. $\|\mathbf{v}\|=\sqrt{3^2+4^2}=5$ , so $\mathbf{u}=\left(\tfrac{3}{5},\tfrac{4}{5}\right)$ .

Step 2 — Compute the gradient.

\frac{\partial f}{\partial x}=2xy,\qquad \frac{\partial f}{\partial y}=x^2+6y.

At $(1,2)$ : $\partial f/\partial x = 2(1)(2)=4$ and $\partial f/\partial y = 1^2+6(2)=13$ . So $\nabla f(1,2)=(4,13)$ .

Step 3 — Apply equation [3].

D_{\mathbf{u}}f(1,2)=\nabla f\cdot\mathbf{u} = 4\cdot\tfrac{3}{5} + 13\cdot\tfrac{4}{5} = \tfrac{12}{5}+\tfrac{52}{5}=\tfrac{64}{5}.

(Since $f$ is a polynomial, it is $C^1$ everywhere, so equation [3] is valid.)

Answer:

\boxed{D_{\mathbf{u}}f(1,2) = \frac{64}{5} = 12.8 \ \text{(units of } f \text{ per unit length)}}

ExampleExample 2 — Steepest Ascent and a Temperature Field

Given: A metal plate has temperature $T(x,y)=100 - x^2 - 2y^2$ (°C), with $x,y$ in centimeters. Find the direction of fastest temperature increase at $(2,1)$ and the maximum rate.

Step 1 — Gradient.

\frac{\partial T}{\partial x}=-2x,\qquad \frac{\partial T}{\partial y}=-4y \;\Rightarrow\; \nabla T(2,1)=(-4,-4).

Step 2 — Direction of steepest ascent (equation [5]) is along $\nabla T$ :

\mathbf{u}=\frac{(-4,-4)}{\|(-4,-4)\|}=\frac{(-4,-4)}{4\sqrt 2}=\left(-\tfrac{1}{\sqrt 2}, -\tfrac{1}{\sqrt 2}\right).

Step 3 — Maximum rate is $\|\nabla T\|$ :

\|\nabla T(2,1)\|=\sqrt{(-4)^2+(-4)^2}=\sqrt{32}=4\sqrt 2 \approx 5.66.

Answer:

\boxed{\text{Steepest ascent toward } \left(-\tfrac{1}{\sqrt2},-\tfrac{1}{\sqrt2}\right),\quad \text{rate } = 4\sqrt2 \approx 5.66\ \tfrac{\text{°C}}{\text{cm}}}

Heat flows opposite to this, along $\left(\tfrac{1}{\sqrt2},\tfrac{1}{\sqrt2}\right)$ (down the gradient).

⚠️ Common mistake

"If all partial derivatives exist, the function is differentiable." FALSE. Partials only probe the coordinate-axis directions. "If all directional derivatives exist (in every direction), the function is differentiable." ALSO FALSE — and even worse, such a function need not be continuous. Differentiability requires the single 2D limit [2] to vanish, which is strictly stronger than the existence of any collection of 1D limits. The correct sufficient condition is continuous partial derivatives ( $C^1$ ), not mere existence.

CounterexampleDirectional Derivatives Exist Everywhere, Yet Not Differentiable / Not Continuous

Define

f(x,y) = \begin{cases} \dfrac{x^2 y}{x^4 + y^2}, & (x,y)\neq(0,0),\\[6pt] 0, & (x,y)=(0,0). \end{cases}

Directional derivatives all exist at the origin. For a unit direction $\mathbf{u}=(a,b)$ with $b\neq 0$ :

D_{\mathbf{u}}f(0,0)=\lim_{h\to0}\frac{1}{h}\cdot\frac{(ha)^2(hb)}{(ha)^4+(hb)^2} = \lim_{h\to0}\frac{a^2 b\,h^3}{h^2(h^2 a^4 + b^2)} = \frac{a^2 b}{b^2}=\frac{a^2}{b}.

For $b=0$ (the $x$ -axis), $f(h,0)=0$ , so $D_{\mathbf{u}}f(0,0)=0$ . Every directional derivative exists.

But $f$ is not even continuous at the origin. Approach along the parabola $y=x^2$ :

f(x,x^2)=\frac{x^2\cdot x^2}{x^4 + x^4}=\frac{x^4}{2x^4}=\frac12 \neq 0 = f(0,0).

The limit depends on the path, so $f$ is discontinuous — hence (by equation [4]) not differentiable, despite all directional derivatives existing.

Moral: the directional derivatives, even if they existed and satisfied $D_{\mathbf{u}}f = \nabla f\cdot\mathbf{u}$ , would not guarantee differentiability. Straight-line probing misses the parabolic ridge.

CounterexamplePartials Exist but Function is Discontinuous

A simpler classic:

g(x,y) = \begin{cases} \dfrac{xy}{x^2+y^2}, & (x,y)\neq(0,0),\\ 0, & (x,y)=(0,0). \end{cases}

Both partials exist at the origin ( $g(x,0)=g(0,y)=0$ , so $g_x(0,0)=g_y(0,0)=0$ ), yet $g(t,t)=\tfrac{1}{2}\to\tfrac12\neq 0$ along $y=x$ . So $g$ is discontinuous, hence not differentiable, even though both partials exist. Existence of partials says nothing about differentiability.

connection

Directional derivatives and differentiability sit at the hub of multivariable analysis. The gradient they define is the workhorse of Gradient Descent in optimization and machine learning. Equation [3] is the seed of the Multivariable Chain Rule, which expresses $\frac{d}{dt}f(\mathbf{r}(t)) = \nabla f\cdot\mathbf{r}'(t)$ — a directional derivative along a curve. The condition [2] generalizes directly to the Total Derivative (Jacobian) for vector-valued maps $\mathbb{R}^n\to\mathbb{R}^m$ , and onward to the Fréchet Derivative in Banach spaces, where the directional version becomes the Gâteaux Derivative. Geometrically, the vanishing of all directional derivatives at an interior extremum gives the first-order condition $\nabla f=\mathbf{0}$ , linking to Lagrange Multipliers and critical-point theory. The orthogonality of $\nabla f$ to level sets connects to Tangent Planes and Linear Approximation and the Implicit Function Theorem.

🔬 Where this appears

⏳ History

- 1797 — Lagrange: Théorie des fonctions analytiques develops partial derivatives systematically and introduces the notation

f'

, treating derivatives as coefficients of Taylor expansions.

- 1821 — Cauchy: Cours d'analyse puts limits and continuity on rigorous footing, giving the modern definition of the derivative via difference quotients — the basis of equation [1].

- 1840s–1870s — Weierstrass: The

\varepsilon

–

\delta

formulation of limits makes precise what "approximation to first order" means, enabling the rigorous remainder condition [2]; his pathological functions warn against naive intuition.

- 1873 — Hamilton & the nabla: The operator

\nabla

, introduced by Hamilton and popularized by Tait, gives the gradient its modern symbol and vector-calculus identity.

- 1881 — Gibbs & Heaviside: Develop vector analysis as we know it, formalizing

\nabla f

, dot products, and the directional-derivative formula [3] for physics applications.

- 1887 — Peano: Provides explicit counterexamples showing existence of partials does not imply differentiability or continuity, sharpening the conceptual distinction.

- 1911–1925 — Fréchet: Defines the (strong) derivative for general normed spaces via condition [2], establishing the modern abstract notion of differentiability.

- 1913 — Gâteaux: Introduces the weaker directional-derivative-style derivative in function spaces, posthumously named the Gâteaux derivative, formalizing the directional vs. total distinction at the highest level.

- 20th century — optimization era: Cauchy's steepest-descent idea (1847) blossoms into modern numerical optimization and, ultimately, deep-learning training.

open-problem

Several deep issues hover around this seemingly elementary topic. (1) The gap between Gâteaux and Fréchet differentiability is enormous in infinite dimensions: a functional can be Gâteaux-differentiable everywhere yet Fréchet-differentiable nowhere, and characterizing where Lipschitz functions on Banach spaces are differentiable is the subject of deep results (Preiss's theorem, 1990) and still-open questions in the geometry of Banach spaces. (2) Differentiability of Lipschitz functions: Rademacher's theorem guarantees a Lipschitz $f:\mathbb{R}^n\to\mathbb{R}$ is differentiable almost everywhere, but the structure of the (measure-zero) non-differentiability set is subtle — related to open problems on "universal differentiability sets." (3) Weak derivatives and Sobolev spaces replace pointwise directional derivatives with distributional ones, the foundation of modern PDE theory, where "differentiability" is reinterpreted entirely. (4) Even in finite dimensions, finding clean necessary-and-sufficient pointwise conditions for differentiability (beyond the sufficient $C^1$ criterion) remains genuinely delicate; existence of all directional derivatives plus continuity still does not suffice in general.

Directional Derivatives and Differentiability

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also