The Gradient and Directional Derivatives

By the end you should be able to: Compute the gradient of a multivariable function, use it to find directional derivatives, and explain why the gradient points in the direction of steepest ascent and is normal to level curves.

Partial derivatives give the slope due east and due north. What about north-east? Or any other direction?

Collecting the partials into a vector answers all of them at once.

The gradient

\nabla f = \left\langle \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \right\rangle

Read "grad $f$ " or "del $f$ ". Input a point, output a vector.

For $f(x,y) = x^2 + 3xy$ :

f_x = 2x + 3y, \qquad f_y = 3x \qquad\Longrightarrow\qquad \nabla f = \langle 2x+3y,\; 3x \rangle

At the point $(1,2)$ :

\nabla f(1,2) = \langle 2(1)+3(2),\; 3(1)\rangle = \langle 8, 3 \rangle

Directional derivatives

The rate of change in the direction of a unit vector $\mathbf{u} = \langle u_1,u_2\rangle$ is

D_{\mathbf u}f = \nabla f \cdot \mathbf{u} = f_x u_1 + f_y u_2

The vector must be a unit vector. If you are given a direction of arbitrary length, divide by its magnitude first. Skipping this is the single most common error in this topic, and it scales the answer wrongly.

For the direction $\langle 3,4\rangle$ , whose length is $\sqrt{9+16}=5$ :

\mathbf{u} = \left\langle \tfrac35, \tfrac45 \right\rangle = \langle 0.6,\, 0.8\rangle

D_{\mathbf u}f(1,2) = \langle 8,3\rangle \cdot \langle 0.6,0.8\rangle = 4.8 + 2.4 = 7.2

Sanity checks the partials give you free

Taking $\mathbf{u} = \langle 1,0\rangle$ gives $D_{\mathbf u}f = f_x$ , and $\langle 0,1\rangle$ gives $f_y$ . The partial derivatives are just directional derivatives along the axes — a special case, not a separate idea.

Why the gradient points uphill

Write the dot product using the angle $\theta$ between $\nabla f$ and $\mathbf u$ :

D_{\mathbf u}f = |\nabla f|\,|\mathbf u|\cos\theta = |\nabla f|\cos\theta

since $|\mathbf u| = 1$ . The only thing you control is $\theta$ , and $\cos\theta$ is largest when $\theta = 0$ .

Direction	$\theta$	Rate of change
Along $\nabla f$	$0$	$+
Against $\nabla f$	$180°$	$-
Perpendicular to $\nabla f$	$90°$	$0$ — along a level curve

Three facts fall out of one line of algebra:

$\nabla f$ points in the direction of steepest increase
$|\nabla f|$ is that steepest rate
$\nabla f$ is perpendicular to the level curve through the point

The third fact, from the previous topic

Moving along a level curve keeps $f$ constant, so the rate of change is zero. Zero rate means $\cos\theta = 0$ , so $\theta = 90°$ . The gradient is normal to the contour — which is why, on a map, the steepest way up a hill is at right angles to the contour lines.

Where it is used

Gradient descent, the workhorse of machine learning, is this fact turned into an algorithm: to minimise a function, repeatedly step in the direction $-\nabla f$ . Module 4 makes it explicit.