Phase 4 · Deep learning · Neural networks
The forward pass
Follow numbers through a network to a prediction.
Open this lesson Create a free account
Why it matters
The forward pass is what a network does every time you use it: numbers in, prediction out. When an LLM writes a word, it runs one forward pass through all its layers and ends with a score (a logit) for every token in its vocabulary. Being able to follow the numbers and the shapes is the skill you'll use to debug every model you build from here on.
What you'll be able to do
- Trace one example through a network: weighted sums, activations, logits, probabilities
- Run a whole batch through a network with matrix multiplications and track every shape
- Turn logits into probabilities with a numerically stable softmax
- Compute the cross-entropy loss that training will try to reduce
Before this lesson
Next in this phase: Computation graphs