Glossary · Phase 6 · Modern LLM engineering
Draft model
In speculative decoding, the small fast model that guesses the next few tokens for the big model to check.
Learn it properly
Speculative decoding · Draft with a small model, verify with a big one. About 35 minutes.
Other terms from the same lesson
- lossless: Changing how something is computed without changing the result at all, such as speculative decoding producing exactly what the big model would have.
- speculative decoding: Speeding up generation by letting a small model draft several tokens and a big model check them at once.