Glossary · Phase 6 · Modern LLM engineering
Catastrophic forgetting
When fine-tuning on a narrow task makes a model lose skills it had before, because the same weights get overwritten.
Learn it properly
Supervised fine-tuning · Teach behavior from example conversations. About 40 minutes.
Other terms from the same lesson
- demonstration: In fine-tuning, an example conversation showing exactly the reply you want the model to learn to give.
- SFT: Supervised fine-tuning: training on example prompts with good answers, so a model learns to respond helpfully.