Partial Derivatives, Gradients & Chain Rule

Extend derivatives to functions with multiple inputs and understand the gradient as the direction in which a function increases most quickly.

By the end of this lesson

You will be able to

Calculate a partial derivative while holding other inputs constant.
Construct a gradient from partial derivatives.
Explain why the negative gradient is a local descent direction.
Apply the chain rule to a simple composition.

Differentiate one input at a time

For a function L(w, b), the partial derivative with respect to w asks how L changes as w moves while b is kept fixed. The partial derivative with respect to b does the opposite. For L(w, b) = w² + 3wb + 2b², the two partial derivatives are 2w + 3b and 3w + 4b.

def partial_w(w, b):
    return 2 * w + 3 * b

def partial_b(w, b):
    return 3 * w + 4 * b

The gradient collects every partial derivative

The gradient is a vector containing all partial derivatives: ∇L = [∂L/∂w, ∂L/∂b]. It points in the direction of steepest local increase. The negative gradient points in the direction of steepest local decrease for a sufficiently small step. The chain rule explains how a derivative passes through a composition: if one function feeds another, the rates multiply along that path.

import numpy as np

grad = np.array([partial_w(1.0, 2.0), partial_b(1.0, 2.0)])
print(grad)  # [8. 11.]

Apply what you learned

Exercise

Calculate a two-parameter gradient

For a small quadratic function, calculate both partial derivatives and assemble the gradient at a chosen point.

Use L(w, b) = w**2 + 3*w*b + 2*b**2.
Differentiate with respect to w while treating b as constant.
Differentiate with respect to b while treating w as constant.
Evaluate both partial derivatives at (w, b) = (1, 2).
Approximate each partial derivative with a finite difference that changes only one input at a time.

Done when

Your gradient at (1, 2) is [8, 11], and each finite-difference estimate is close to its analytic partial derivative.

Free references

Study resources

3 links

01
Introduction to partial derivatives Article
Read the introductory definition and first example; focus on holding all other inputs constant.

Done
02
The gradient Article
Read the geometric interpretation of the gradient and its direction of steepest increase.

Done
03
Visualizing the chain rule and product rule | Chapter 4 Video
Watch the chain-rule portion of the chapter. The product-rule section is optional for this module.

Done

Check your understanding

Self-test

2 questions

Question 1

What is held fixed when calculating ∂L/∂w for L(w, b)?
Question 2

Which direction does the gradient point?

Completion check

Move on when

You can calculate a gradient for a two-parameter function and explain how the negative gradient relates to reducing a loss.

Mark passed and continue Back to module