Index Ai Academy
Glossary · Phase 5 · Transformers

Multi-head attention

Also written: attention head, attention heads

Running several attention patterns side by side, so the model can track different relationships at once.

Learn it properly

Multi-head attention · Several attention patterns in parallel. About 35 minutes.

Open the lesson Create a free account