Glossary · Phase 1 · Building with LLMs
A/B test
Show two versions to two random groups and compare the results, to learn which one really works better.
Learn it properly
Evals · Measure output quality with test sets instead of vibes. About 30 minutes.
Other terms from the same lesson
- baseline: A simple reference result to beat, such as always guessing the most common answer. A model is only useful if it does better.
- eval: A set of test cases with expected results, used to measure how well an AI system does its job.
- held-out: Data kept aside and never used for training, so you can test the model on examples it hasn't seen.