Phase 6 · Modern LLM engineering · Post-training
Training reasoning models
Reinforcement learning on verifiable problems.
Open this lesson Create a free account
Why it matters
The biggest change in models since 2024 is reasoning: models that think at length before answering, and are far better at math, code and multi-step problems as a result. The training recipe behind them is surprisingly simple, reinforcement learning against answers that can be checked automatically. Knowing how it works explains what these models are good at, why "thinking effort" settings exist (Phase 1), and where they still go wrong.
What you'll be able to do
- Explain reinforcement learning with verifiable rewards and why it scales without human labels
- Implement GRPO's group-relative advantage and train a small policy with it
- Recognize reward hacking against weak checkers
- Relate test-time compute (sampling, voting, thinking longer) to accuracy
Before this lesson
Next in this phase: Prompting vs RAG vs fine-tuning