Teacher model

Also written: teacher models

In distillation, the large model whose answers or probabilities a smaller student model learns to copy.

Other terms from the same lesson

• distillation: Training a small model to imitate a big one, keeping much of its skill at lower cost.
• soft labels: A teacher's full spread of probabilities over the answers, which carries more information than just the single right answer (a hard label).
• student model: In distillation, the smaller model trained to copy a larger teacher model.