Glossary · Phase 5 · Transformers
Multi-head attention
Running several attention patterns side by side, so the model can track different relationships at once.
Learn it properly
Multi-head attention · Several attention patterns in parallel. About 35 minutes.