Phase 3 · Machine learning · Foundations
Train, validation and test sets
Hold data back so you can trust your results.
Open this lesson Create a free account
Why it matters
A model that memorizes its training data can score perfectly on it and be useless on new customers. The only honest measure of a model is how it does on data it has never seen, so you hold some data back. This is the same principle as an eval set for prompts in Phase 1, and the same mistake (tuning on your test set) ruins both.
What you'll be able to do
- Explain why a model must be scored on data it never trained on
- Give train, validation and test sets their separate jobs
- Split randomly with a seed, stratify by label, or split by time
- Recognize the ways test data leaks into training
Before this lesson
Next in this phase: Linear regression