Phase 3 · Machine learning · Models
Learning rate
Step size, convergence and divergence.
Open this lesson Create a free account
Why it matters
The learning rate is the single setting that most often makes training succeed or fail. Too small and training crawls; too large and it explodes. Every training recipe for LLMs specifies a learning rate and a schedule (warm up, then decay), and when a fine-tuning run goes wrong, it's the first thing people check. A few minutes of intuition here saves hours of confused retraining.
What you'll be able to do
- Diagnose a learning rate that's too small, too large or about right
- Find a good learning rate with a quick sweep
- Explain learning-rate schedules: step decay, warmup and cosine decay
- Connect feature scaling to which learning rates are usable
Before this lesson
Next in this phase: Logistic regression