Phase 5 · Transformers · Language modeling
Sampling strategies
Temperature, top-k and top-p: turning probabilities into text.
Open this lesson Create a free account
Why it matters
A language model only outputs probabilities. How you pick from them, the decoding strategy, decides whether the same model sounds repetitive, sensible or unhinged. These are the knobs behind "temperature" and "top-p" in many APIs and open-model tools, and understanding them explains why the same prompt gives different answers and why turning randomness all the way down isn't always better.
What you'll be able to do
- Turn a model's next-token probabilities into a choice: greedy versus sampling
- Reshape the distribution with temperature
- Cut off the unlikely tail with top-k and top-p (nucleus) sampling
- Know which of these settings today's APIs expose
Before this lesson
Next in this phase: Neural language models