Phase 2 · Math, when you need it · Calculus
The chain rule
Derivatives of functions inside functions: the heart of backprop.
Open this lesson Create a free account
Why it matters
A neural network is functions inside functions inside functions: layer after layer, then a loss at the end. To train it you need the derivative of that loss with respect to every weight, however deep. The chain rule is how: multiply the local rates along the path from the weight to the loss. Backpropagation, the algorithm that trains every modern model, is nothing more than the chain rule organized so that each step is computed once.
What you'll be able to do
- Differentiate a function inside a function by multiplying rates
- Follow a computation graph forward (values) and backward (derivatives)
- Compute the gradient of a single neuron's loss by hand
- Explain backpropagation as the chain rule applied from the loss backwards
Before this lesson
Next in this phase: Partial derivatives