Partial Derivatives and the Gradient

⚡ Key Takeaways

One-line statement: A partial derivative measures the rate of change of a multivariable function with respect to one variable while holding all others fixed; the gradient assembles these partials into a vector pointing in the direction of steepest increase. Key result in words: $\nabla f$ is perpendicular to level sets and its magnitude gives the maximum directional rate of change. Where it applies: Optimization, physics (fields, forces), machine learning (gradient descent), thermodynamics, differential geometry. Who/when: Concept matured through Euler, Clairaut, and Jacobi in the 18th–19th centuries; vector notation ∇ popularized by Hamilton and Gibbs.

💡 The Core Insight

Differentiation in many dimensions splits into two complementary objects. Partial derivatives are scalar slopes along coordinate axes. The gradient is the vector that organizes them so that, for any unit direction $\mathbf{u}$ , the rate of change of $f$ in that direction is simply $\nabla f \cdot \mathbf{u}$ . The single dot product collapses an infinite family of directional questions into one linear algebra operation — and immediately reveals that the steepest ascent direction is $\nabla f$ itself.

DefinitionPartial Derivative

Let $f:\mathbb{R}^n \to \mathbb{R}$ and let $\mathbf{a}=(a_1,\dots,a_n)$ . The partial derivative of $f$ with respect to $x_i$ at $\mathbf{a}$ is

1

\quad \frac{\partial f}{\partial x_i}(\mathbf{a}) = \lim_{h\to 0}\frac{f(a_1,\dots,a_i+h,\dots,a_n)-f(a_1,\dots,a_i,\dots,a_n)}{h}

Derivation hint: This is the ordinary single-variable difference quotient applied to the restricted function $g(t)=f(a_1,\dots,t,\dots,a_n)$ at $t=a_i$ ; all other slots are frozen.

Symbols: $f$ is the scalar field; $x_i$ is the $i$ -th input coordinate; $\mathbf{a}$ is the evaluation point; $h$ is the perturbation along axis $i$ . Equivalent notations: $\partial_{x_i} f$ , $f_{x_i}$ , $D_i f$ .

DefinitionThe Gradient

For a differentiable $f:\mathbb{R}^n\to\mathbb{R}$ , the gradient is the vector of all first partials:

2

\quad \nabla f(\mathbf{a}) = \left(\frac{\partial f}{\partial x_1}(\mathbf{a}),\,\frac{\partial f}{\partial x_2}(\mathbf{a}),\,\dots,\,\frac{\partial f}{\partial x_n}(\mathbf{a})\right)

Derivation hint: Stack the $n$ scalars from [1] (one per coordinate axis) into an ordered tuple; the result is a vector in the same space as the inputs.

Symbols: $\nabla$ ("del" or "nabla") is the gradient operator; $\nabla f$ is a vector field assigning a vector to each point. In Cartesian coordinates $\nabla = \big(\partial_{x_1},\dots,\partial_{x_n}\big)$ .

DefinitionDirectional Derivative

The rate of change of $f$ at $\mathbf{a}$ in the direction of a unit vector $\mathbf{u}$ is

3

\quad D_{\mathbf{u}}f(\mathbf{a}) = \lim_{h\to 0}\frac{f(\mathbf{a}+h\mathbf{u})-f(\mathbf{a})}{h} = \nabla f(\mathbf{a})\cdot \mathbf{u}

Derivation hint: Apply the chain rule to $\phi(h)=f(\mathbf{a}+h\mathbf{u})$ and evaluate $\phi'(0)$ ; the chain rule produces $\sum_i \partial_{x_i}f \cdot u_i = \nabla f \cdot \mathbf{u}$ .

Symbols: $\mathbf{u}$ is a unit vector ( $\|\mathbf{u}\|=1$ ); $D_{\mathbf{u}}f$ is the directional derivative. The dot-product form holds only when $f$ is differentiable (existence of partials alone is insufficient).

TheoremSteepest Ascent and Orthogonality to Level Sets

Let $f$ be differentiable at $\mathbf{a}$ with $\nabla f(\mathbf{a})\neq \mathbf{0}$ . Then: 1. The maximum value of $D_{\mathbf{u}}f(\mathbf{a})$ over all unit $\mathbf{u}$ equals $\|\nabla f(\mathbf{a})\|$ , attained when $\mathbf{u}=\nabla f(\mathbf{a})/\|\nabla f(\mathbf{a})\|$ .

4

\quad \max_{\|\mathbf{u}\|=1} D_{\mathbf{u}}f(\mathbf{a}) = \|\nabla f(\mathbf{a})\|

Derivation hint: From [3], $D_{\mathbf{u}}f=\|\nabla f\|\,\|\mathbf{u}\|\cos\theta=\|\nabla f\|\cos\theta$ ; this is maximized at $\theta=0$ . 2. $\nabla f(\mathbf{a})$ is orthogonal to the level set $\{f=f(\mathbf{a})\}$ at $\mathbf{a}$ .

5

\quad \nabla f(\mathbf{a})\cdot \mathbf{v} = 0 \quad \text{for every tangent vector } \mathbf{v} \text{ to the level set at } \mathbf{a}

Derivation hint: Parametrize a curve $\mathbf{r}(t)$ lying in the level set, so $f(\mathbf{r}(t))=c$ ; differentiate using the chain rule to get $\nabla f\cdot \mathbf{r}'(t)=0$ .

ProofGradient gives directional derivative and steepest ascent

Step 1 — Reduce to one variable. Fix $\mathbf{a}$ and a unit direction $\mathbf{u}=(u_1,\dots,u_n)$ . Define $\phi(h)=f(\mathbf{a}+h\mathbf{u})$ , a function of the single real variable $h$ .

Step 2 — Apply the multivariable chain rule. If $f$ is differentiable at $\mathbf{a}$ , then

6

\quad \phi'(0)=\sum_{i=1}^n \frac{\partial f}{\partial x_i}(\mathbf{a})\,\frac{d(a_i+hu_i)}{dh}\Big|_{h=0}=\sum_{i=1}^n \frac{\partial f}{\partial x_i}(\mathbf{a})\,u_i.

Derivation hint: The components of $\mathbf{a}+h\mathbf{u}$ are $a_i+hu_i$ with derivative $u_i$ ; sum the chain-rule contributions.

Step 3 — Recognize the dot product. The sum in [6] is precisely $\nabla f(\mathbf{a})\cdot\mathbf{u}$ , establishing [3].

Step 4 — Maximize over directions. Write the dot product as

7

\quad \nabla f(\mathbf{a})\cdot\mathbf{u}=\|\nabla f(\mathbf{a})\|\cos\theta,

Derivation hint: Use the geometric definition of dot product with $\|\mathbf{u}\|=1$ ; $\theta$ is the angle between $\nabla f$ and $\mathbf{u}$ . where $\theta$ is the angle between $\nabla f(\mathbf{a})$ and $\mathbf{u}$ . Since $\cos\theta\in[-1,1]$ , the maximum $\|\nabla f(\mathbf{a})\|$ occurs at $\theta=0$ (steepest ascent) and the minimum $-\|\nabla f(\mathbf{a})\|$ at $\theta=\pi$ (steepest descent). This proves [4]. The orthogonality claim [5] follows because tangent directions to the level set correspond to $\theta=\pi/2$ where the directional derivative vanishes. $\blacksquare$

∎

⚠️ Differentiability is required, not just existence of partials

Equations [3]–[5] assume $f$ is differentiable at $\mathbf{a}$ , meaning $f$ admits a good linear approximation. The mere existence of all partial derivatives does not guarantee differentiability, and in such pathological cases the formula $D_{\mathbf{u}}f=\nabla f\cdot\mathbf{u}$ can fail. A standard sufficient condition: if all partials exist and are continuous in a neighborhood of $\mathbf{a}$ (i.e. $f\in C^1$ ), then $f$ is differentiable there.

CG

Carl Gustav Jacob Jacobi

1804–1851

Jacobi systematized partial differentiation through his work on the theory of multiple-variable transformations, introducing the Jacobian determinant that bears his name and embedding partial derivatives into a rigorous matrix framework. His treatment of functional dependence and changes of variable made the array of partial derivatives — what we now call the Jacobian matrix, whose single-output special case is the gradient — a first-class mathematical object. Jacobi's emphasis on the structured organization of derivatives directly anticipated the modern vector gradient and the broader machinery of differential calculus on $\mathbb{R}^n$ .

🧠 Intuition

Picture $f(x,y)$ as the elevation of a mountainous terrain over a map. A partial derivative $\partial f/\partial x$ is the slope you feel if you walk due east, ignoring north–south tilt — it is one specific compass-aligned slope. But terrain doesn't care about compass directions; the steepest uphill might point northeast. The gradient is the arrow on your map that points directly uphill in the steepest possible direction, and its length tells you how steep that climb is. Crucially, the gradient is built only from the east–west and north–south slopes, yet it correctly identifies the best diagonal direction — because any direction's slope is a weighted blend of the axis slopes, and that blend is exactly a dot product. Water flowing downhill always runs anti-parallel to the gradient, and contour lines on a topographic map (level sets) always cross the gradient at right angles.

Figure

Gradient vector perpendicular to level curves, pointing toward steepest ascent

ExampleGradient of a paraboloid and steepest ascent

Given: $f(x,y)=x^2+3y^2$ , evaluate at $\mathbf{a}=(2,1)$ . Find $\nabla f$ , the steepest-ascent direction, and the directional derivative toward $\mathbf{u}=\tfrac{1}{\sqrt 5}(1,2)$ .

Step 1 — Partial derivatives.

8

\quad \frac{\partial f}{\partial x}=2x,\qquad \frac{\partial f}{\partial y}=6y.

Derivation hint: Treat $y$ constant for $\partial_x$ (so $3y^2$ vanishes), and $x$ constant for $\partial_y$ .

Step 2 — Gradient at the point. $\nabla f(2,1)=(2\cdot 2,\,6\cdot 1)=(4,6).$

Step 3 — Steepest ascent magnitude.

9

\quad \|\nabla f(2,1)\|=\sqrt{4^2+6^2}=\sqrt{52}=2\sqrt{13}\approx 7.21.

Derivation hint: Euclidean norm of $(4,6)$ .

Step 4 — Directional derivative toward $\mathbf{u}$ .

10

\quad D_{\mathbf{u}}f=(4,6)\cdot\tfrac{1}{\sqrt5}(1,2)=\tfrac{4+12}{\sqrt5}=\tfrac{16}{\sqrt5}\approx 7.16.

Derivation hint: Dot product of gradient with the unit vector, per [3].

Answer: $\boxed{\nabla f(2,1)=(4,6),\ \ \text{steepest slope}=2\sqrt{13}\approx 7.21,\ \ D_{\mathbf u}f\approx 7.16}$ (units: output-per-unit-length). The chosen $\mathbf{u}$ is nearly aligned with the steepest direction $(4,6)/\sqrt{52}\approx(0.55,0.83)$ , hence $7.16\approx 7.21$ .

ExampleTemperature gradient — heat flux direction

Given: A metal plate has temperature $T(x,y)=100 - x^2 - 2y^2$ (in °C, $x,y$ in cm). A bug sits at $(3,2)$ . Which way is it coolest, and how fast does temperature drop there?

Step 1 — Gradient. $\nabla T=(-2x,-4y)$ , so $\nabla T(3,2)=(-6,-8).$ This points toward increasing temperature.

Step 2 — Coolest direction = negative gradient.

11

\quad -\nabla T(3,2)=(6,8),\qquad \text{unit form }\tfrac{1}{10}(6,8)=(0.6,0.8).

Derivation hint: Steepest decrease is anti-parallel to the gradient; normalize $(6,8)$ whose norm is $10$ .

Step 3 — Rate of decrease.

12

\quad \|\nabla T(3,2)\|=\sqrt{(-6)^2+(-8)^2}=\sqrt{100}=10\ \text{°C/cm}.

Derivation hint: Norm of $(-6,-8)$ .

Answer: $\boxed{\text{Coolest direction }(0.6,0.8),\ \text{temperature falls at }10\ \text{°C/cm}}$ . By Fourier's law, heat flux $\mathbf{q}=-k\nabla T$ flows in direction $(6,8)$ , from hot toward cold.

⚠️ Frequent errors with partials and gradients

The gradient is not a slope (scalar); it is a vector. Its magnitude is the maximal slope.
Do not confuse $\nabla f$ (gradient, input on a scalar field) with the divergence $\nabla\cdot\mathbf{F}$ or curl $\nabla\times\mathbf{F}$ , which act on vector fields and produce different objects.
Mixed partials need not be equal in pathological cases. Clairaut's theorem guarantees $\partial_{xy}f=\partial_{yx}f$ only when those mixed partials are continuous.
Existence of partials ≠ continuity ≠ differentiability. All four properties (partials exist, continuity, differentiability, $C^1$ ) are genuinely distinct.
The dot-product formula $D_{\mathbf u}f=\nabla f\cdot\mathbf u$ requires $\mathbf u$ to be a unit vector; using a non-unit vector silently rescales the rate.

CounterexamplePartials exist but

f

is not even continuous

Consider

13

\quad f(x,y)=\begin{cases}\dfrac{xy}{x^2+y^2}, & (x,y)\neq(0,0)\\[2mm] 0,& (x,y)=(0,0).\end{cases}

Derivation hint: Along $y=mx$ , $f=\frac{m}{1+m^2}$ depends on $m$ , so the limit at the origin is path-dependent. Both partials exist at the origin: $f_x(0,0)=\lim_{h\to0}\frac{f(h,0)-0}{h}=0$ and likewise $f_y(0,0)=0$ . Yet $f$ is not continuous at $(0,0)$ — approaching along $y=x$ gives $f\to\tfrac12$ , but along the axes $f=0$ . Hence the partials existing tells us nothing about continuity, let alone differentiability. The naive formula $D_{\mathbf u}f=\nabla f\cdot\mathbf u=0$ would falsely predict zero change in every direction.

connection

The gradient is the transpose / vector form of the <span class="wiki-link-missing">Jacobian Matrix</span> for the special case of a scalar-valued function. It is the central ingredient of the Multivariable Chain Rule, which expresses $\tfrac{d}{dt}f(\mathbf r(t))=\nabla f\cdot \mathbf r'(t)$ . It underlies the Method of Lagrange Multipliers, where constrained optima require $\nabla f=\lambda\nabla g$ (gradients parallel). In vector calculus, $\nabla$ generalizes to the Divergence and Curl operators, and the gradient is the first member of the differential-operator family unified by the Exterior Derivative. The condition $\nabla f=\mathbf 0$ defines Critical Points and the Second Derivative Test.

🔬 Where this appears

⏳ History

- 1740s — Euler develops systematic partial differentiation in the study of functions of several variables and the calculus of variations.

- 1740 — Clairaut establishes the equality of mixed partial derivatives under continuity (now Clairaut's / Schwarz's theorem), in work on the shape of the Earth.

- 1770s — Lagrange introduces multiplier methods for constrained optimization, implicitly using gradient parallelism.

- 1797 — Lagrange popularizes prime/subscript derivative notation in Théorie des fonctions analytiques.

- 1841 — Jacobi publishes his theory of functional determinants, formalizing arrays of partial derivatives (the Jacobian).

- 1843 — Hamilton introduces the operator

\nabla

("nabla") in connection with quaternions.

- 1873 — Maxwell employs and refines

\nabla

notation in his Treatise on Electricity and Magnetism, including the gradient.

- 1880s — Gibbs and Heaviside develop modern vector calculus, cementing

\nabla f

, divergence, and curl as standard notation taught today.

- 1847 — Cauchy describes the method of steepest descent, the conceptual ancestor of gradient-based optimization.

open-problem

Several conceptual subtleties extend beyond the introductory picture. (1) Coordinate dependence: the naive component definition of $\nabla f$ is correct only in orthonormal Cartesian coordinates; in curvilinear or non-Euclidean settings the gradient requires the metric tensor, $(\nabla f)^i = g^{ij}\partial_j f$ , distinguishing the gradient vector from the differential $df$ (a covector). This distinction — vectors versus covectors — is invisible in $\mathbb{R}^n$ with the standard metric but essential in differential geometry and general relativity. (2) Non-smoothness: for non-differentiable convex functions (e.g. $\|\mathbf x\|_1$ in machine learning) the gradient is replaced by the subgradient, and optimization theory must handle set-valued generalizations. (3) Infinite dimensions: generalizing the gradient to function spaces leads to the Fréchet and Gâteaux derivatives, with delicate existence questions central to the calculus of variations and PDE theory.

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also