Glossary · Phase 6 · Modern LLM engineering
GRPO
Group relative policy optimization: sample several answers to one problem and push up the ones that beat the group's average.
Learn it properly
Training reasoning models · Reinforcement learning on verifiable problems. About 40 minutes.
Other terms from the same lesson
- advantage: In reinforcement learning, how much better one answer scored than expected (in GRPO, than its group's average).
- pass@k: The chance that at least one of k attempts at a problem is correct.
- policy gradient: Training by making high-scoring answers more likely and low-scoring ones less likely, in proportion to how good they were.
- RLVR: Reinforcement learning with verifiable rewards: training on problems whose answers a program can check, such as math or code with tests.
- self-consistency: Asking a model the same question several times and taking the most common answer.
- test-time compute: Extra computation spent while answering, such as thinking longer or trying several answers, rather than during training.