Glossary · Phase 1 · Building with LLMs
Eval
A set of test cases with expected results, used to measure how well an AI system does its job.
Learn it properly
Evals · Measure output quality with test sets instead of vibes. About 30 minutes.
Other terms from the same lesson
- A/B test: Show two versions to two random groups and compare the results, to learn which one really works better.
- baseline: A simple reference result to beat, such as always guessing the most common answer. A model is only useful if it does better.
- held-out: Data kept aside and never used for training, so you can test the model on examples it hasn't seen.