Phase 5 · Transformers · Attention
Queries, keys and values
The three projections behind attention.
Open this lesson Create a free account
Why it matters
The previous lesson's attention could only link similar words. Real attention gives every token three separate learned vectors: what it's looking for, what it offers to others' searches, and what it passes along when chosen. Those three projections are where most of attention's power lives, and they're the Q, K and V you'll see in every transformer diagram and model card.
What you'll be able to do
- Explain the three roles a token plays in attention: query, key and value
- Compute Q, K and V with learned projection matrices
- Show how projections let attention express relations that plain similarity can't
- Connect attention to retrieval as a soft, differentiable lookup
Before this lesson
Next in this phase: Scaled dot-product attention