Glossary · Phase 3 · Machine learning
Calibration
Whether a model's probabilities can be taken at face value: of the customers it gives 30%, about 30% really churn.
Learn it properly
ROC curves and AUC · Judge a classifier across every threshold. About 30 minutes.
Other terms from the same lesson
- AUC: Area under the ROC curve: the chance the model scores a random true case above a random false one. 1 is perfect, 0.5 is guessing.
- average precision: The area under the precision-recall curve: one number for how well a model finds a rare class.
- false positive rate: The share of real negatives the model wrongly flags, such as the share of loyal customers it says will churn.
- precision-recall curve: A chart of precision against recall as the threshold moves. More telling than the ROC curve when the thing you're looking for is rare.
- ROC curve: A chart of how a classifier trades catching more true cases against raising more false alarms.
- true positive rate: The share of real positives the model catches, such as the share of churners it flags. Another name for recall.