Glossary · Phase 6 · Modern LLM engineering
Ablation
Switching off one part of a model, such as an attention head, to see what changes. A big change means that part mattered.
Learn it properly
Interpretability · Looking inside models to see what they compute. About 40 minutes.
Other terms from the same lesson
- interpretability: Research into what is actually happening inside a model's numbers.
- logit lens: Reading a model's in-progress guess at each layer by passing the hidden state straight to the output layer.
- steering: Changing a model's behavior by adding a direction to its hidden states while it runs, without retraining it.
- superposition: When a model packs more features than it has neurons by sharing each neuron across several of them, which makes single neurons hard to read.