MULTIVARIABLE CALCULUS 03

The Gradient

The gradient collects the partial derivatives of a scalar function into a vector that points in the direction of steepest increase.

The main idea

Suppose

\[ f(x,y) \]

is a scalar-valued function.

We already know that

\[ \frac{\partial f}{\partial x} \]

measures change in the \(x\)-direction, while

\[ \frac{\partial f}{\partial y} \]

measures change in the \(y\)-direction.

The gradient combines these into one vector:

\[ \boxed{ \nabla f = \begin{pmatrix} \frac{\partial f}{\partial x}\\[4pt] \frac{\partial f}{\partial y} \end{pmatrix} } \]

1. The symbol \(\nabla\)

The symbol

\[ \nabla \]

is called nabla, and is often read as “del”.

\[ \nabla f \]

is read as

“the gradient of \(f\)”.

2. A simple example

Let

\[ f(x,y)=x^2+y^2. \]

Then

\[ \frac{\partial f}{\partial x}=2x \]

and

\[ \frac{\partial f}{\partial y}=2y. \]

Therefore,

\[ \boxed{ \nabla f = \begin{pmatrix} 2x\\ 2y \end{pmatrix} } \]

3. Evaluate the gradient at a point

\[ (x,y)=(1,2), \]

we have

\[ \nabla f(1,2) = \begin{pmatrix} 2\\ 4 \end{pmatrix}. \]

So the gradient vector at \((1,2)\) points in the direction

\[ \begin{pmatrix} 2\\ 4 \end{pmatrix}. \]

4. What direction does the gradient point?

The gradient points in the direction in which the function increases most rapidly.

\[ \boxed{ \nabla f = \text{direction of steepest increase} } \]

If we move in the opposite direction,

\[ -\nabla f, \]

the function decreases most rapidly.

5. Why the gradient is a vector

For a one-variable function, there is only one input direction.

For a function \(f(x,y)\), there are infinitely many possible directions in the plane.

The gradient captures the local change information in a single vector.

6. Level curves

A level curve satisfies

\[ f(x,y)=c, \]

where \(c\) is constant.

Along a level curve, the value of \(f\) does not change.

Therefore, movement along the curve produces zero first-order change in \(f\).

7. Gradient and level curves

The gradient is perpendicular to the level curve.

\[ \boxed{ \nabla f \perp \text{level curve} } \]

This is one of the most important geometric meanings of the gradient.

8. Example with circular level curves

For

\[ f(x,y)=x^2+y^2, \]

the level curves are

\[ x^2+y^2=c. \]

These are circles centred at the origin.

The gradient is

\[ \nabla f= \begin{pmatrix} 2x\\ 2y \end{pmatrix}. \]

This vector points directly away from the origin.

So it is perpendicular to the circular level curves.

9. Diagram

10. Directional change

Suppose we move from a point in a direction given by a unit vector

\[ \mathbf{u}. \]

The rate of change of \(f\) in that direction is

\[ \boxed{ D_{\mathbf{u}}f = \nabla f\cdot\mathbf{u} } \]

This is called the directional derivative.

11. Why the dot product appears

The dot product measures how strongly the direction of movement aligns with the gradient.

If \(\mathbf{u}\) points in the same direction as \(\nabla f\), the directional derivative is largest.

If \(\mathbf{u}\) is perpendicular to \(\nabla f\), then

\[ \nabla f\cdot\mathbf{u}=0. \]

So there is no first-order change in \(f\) in that direction.

12. Magnitude of the gradient

For

\[ \nabla f = \begin{pmatrix} f_x\\ f_y \end{pmatrix}, \]

its magnitude is

\[ \boxed{ \|\nabla f\| = \sqrt{ \left(\frac{\partial f}{\partial x}\right)^2 + \left(\frac{\partial f}{\partial y}\right)^2 } } \]

This gives the maximum directional rate of increase at that point.

13. Example

For

\[ f(x,y)=x^2+y^2, \]

at \((1,2)\),

\[ \nabla f= \begin{pmatrix} 2\\ 4 \end{pmatrix}. \]

Its magnitude is

\[ \|\nabla f\| = \sqrt{2^2+4^2} = \sqrt{20} = 2\sqrt5. \]

So the maximum local rate of increase is

\[ \boxed{ 2\sqrt5 } \]

14. Gradient in three variables

\[ f(x,y,z) \]

depends on three variables, then

\[ \boxed{ \nabla f = \begin{pmatrix} \frac{\partial f}{\partial x}\\[4pt] \frac{\partial f}{\partial y}\\[4pt] \frac{\partial f}{\partial z} \end{pmatrix} } \]

15. Example in three dimensions

Let

\[ f(x,y,z)=x^2+y^2+z^2. \]

Then

\[ \boxed{ \nabla f = \begin{pmatrix} 2x\\ 2y\\ 2z \end{pmatrix} } \]

The gradient points radially outward from the origin.

16. Gradient and optimisation

For a differentiable function, an interior local maximum or minimum often occurs where

\[ \boxed{ \nabla f=\mathbf{0} } \]

That means all first partial derivatives are zero:

\[ \frac{\partial f}{\partial x}=0, \qquad \frac{\partial f}{\partial y}=0. \]

This extends the one-variable condition

\[ f'(x)=0. \]

17. Gradient descent

Because

\[ \nabla f \]

points toward steepest increase,

\[ -\nabla f \]

points toward steepest decrease.

This leads to the basic optimisation idea

\[ \boxed{ \mathbf{x}_{n+1} = \mathbf{x}_n - \alpha\nabla f(\mathbf{x}_n) } \]

where \(\alpha>0\) is a step size.

This is the basic principle behind gradient descent.

18. Gradient in modelling

Suppose an outcome

\[ R(\beta,\gamma) \]

depends on two parameters.

Then

\[ \nabla R = \begin{pmatrix} \frac{\partial R}{\partial\beta}\\ \frac{\partial R}{\partial\gamma} \end{pmatrix}. \]

This vector describes how sensitive \(R\) is to small changes in both parameters.

19. Partial derivatives versus gradient

A partial derivative gives one directional component of change:

\[ f_x, \qquad f_y. \]

The gradient collects these components:

\[ \boxed{ \nabla f = \begin{pmatrix} f_x\\ f_y \end{pmatrix} } \]

So the gradient is not a new type of derivative separate from partial derivatives. It is a vector built from them.

20. The central idea

\[ \boxed{ \text{partial derivatives} \longrightarrow \text{gradient vector} } \] \[ \boxed{ \nabla f = \text{direction of steepest increase} } \] \[ \boxed{ \|\nabla f\| = \text{maximum local rate of increase} } \]

The gradient turns local rate-of-change information into a geometric direction.