Phase 3 · Machine learning · Evaluation
Cross-validation
Reuse limited data for more reliable estimates.
Open this lesson Create a free account
Why it matters
With 600 customers, a validation set of 180 is small enough that which 180 you happened to pick moves the score by several points. That's enough to crown the wrong model. Cross-validation uses every example for validation once and averages, so the comparison reflects the models instead of the dice. The same worry applies to LLM evals: a prompt that "wins" by two points on 40 examples may not have won anything.
What you'll be able to do
- Show how much a single train/validation split's score depends on luck
- Run k-fold cross-validation and report a mean and spread
- Tune settings with cross-validation while keeping a final test set untouched
- Pick the right splitter for grouped or time-ordered data
Before this lesson
Next in this phase: Clustering