Training with gradient descent

Why it matters

Linear regression has an exact solution, so why learn to train it step by step? Because almost nothing else does. Logistic regression, neural networks and every LLM are trained with the loop you'll write here: shuffle the data, take a mini-batch, compute the gradient of the loss, nudge the weights, repeat for several epochs. Writing it once on a model you can check makes every training script you'll ever read familiar.

What you'll be able to do

• Train a regression model on real data with gradient descent, from scratch
• Use mini-batches, epochs and shuffling the way deep learning does
• Read a training loss curve
• Know when gradient descent is needed and when an exact solution exists

Before this lesson

• Loss functions
• Optimization by walking downhill

Next in this phase: Learning rate