The training loop

Why it matters

Every neural network you'll train, from this lesson's digit classifier to a fine-tuned LLM, is trained by the same five-line loop. The variations (bigger batches, fancier optimizers, schedules, mixed precision) are all modifications of it. Knowing it cold means you can read any training script and see where a problem might be hiding.

What you'll be able to do

• Write the standard PyTorch training loop: zero_grad, forward, loss, backward, step
• Use an optimizer from torch.optim instead of hand-written updates
• Evaluate correctly with model.eval() and torch.no_grad()
• Track training and validation curves and keep the best checkpoint

Before this lesson

• PyTorch autograd and modules
• Learning rate

Next in this phase: Mini-batches and SGD