Phase 4 · Deep learning · Neural networks
Computation graphs
Every calculation as a graph of small operations.
Open this lesson Create a free account
Why it matters
To train a network you need the gradient of the loss with respect to every weight, and a network can have billions of them. Nobody derives those by hand. Instead, frameworks record the calculation as a graph of tiny operations whose derivatives are trivial, then combine them mechanically. This lesson is that graph; the next one runs it backwards.
What you'll be able to do
- Break a calculation into a graph of small operations
- Evaluate a graph in topological order (the forward pass)
- Give every operation its local derivative
- Combine local derivatives with the chain rule, adding up contributions from branches
Before this lesson
Next in this phase: Backpropagation