Policy gradient

Also written: policy gradients

Training by making high-scoring answers more likely and low-scoring ones less likely, in proportion to how good they were.

Other terms from the same lesson

• advantage: In reinforcement learning, how much better one answer scored than expected (in GRPO, than its group's average).
• GRPO: Group relative policy optimization: sample several answers to one problem and push up the ones that beat the group's average.
• pass@k: The chance that at least one of k attempts at a problem is correct.
• RLVR: Reinforcement learning with verifiable rewards: training on problems whose answers a program can check, such as math or code with tests.
• self-consistency: Asking a model the same question several times and taking the most common answer.
• test-time compute: Extra computation spent while answering, such as thinking longer or trying several answers, rather than during training.