Backpropagation

Why it matters

Backpropagation is how every neural network learns, from a two-layer classifier to a frontier LLM. It's the computation-graph idea from the last lesson, organized so one backward sweep produces the gradient for every weight at once. Writing it by hand once, for a real network, makes everything that frameworks do automatically feel concrete rather than magical.

What you'll be able to do

• Run the backward pass: from the loss back through every layer, one local rule at a time
• Write the matrix gradients of a linear layer, ReLU, and softmax with cross-entropy
• Check hand-written gradients against finite differences
• Explain why backpropagation makes training billions of weights feasible

Before this lesson

• Computation graphs

Next in this phase: Build an autograd engine