Glossary · Phase 5 · Transformers
Training loss
The model's average error on the examples it learns from. It should fall as training goes on.
Learn it properly
Build a tiny GPT · Write and train a small GPT from scratch. About 45 minutes.
Other terms from the same lesson
- validation loss: The model's average error on held-back text it never trains on. If it rises while training loss keeps falling, the model is memorizing.