Soft labels

Also written: soft label, soft targets

A teacher's full spread of probabilities over the answers, which carries more information than just the single right answer (a hard label).

Other terms from the same lesson

• distillation: Training a small model to imitate a big one, keeping much of its skill at lower cost.
• student model: In distillation, the smaller model trained to copy a larger teacher model.
• teacher model: In distillation, the large model whose answers or probabilities a smaller student model learns to copy.