Glossary · Phase 3 · Machine learning
Gradient boosting
Adding small decision trees one at a time, each fixing the mistakes left by the ones before.
Learn it properly
Random forests and boosting · Many weak models voting beat one strong one. About 30 minutes.
Other terms from the same lesson
- bagging: Training many models, each on a random resample of the data, and averaging their answers. The averaging cancels out much of each model's random error.
- early stopping: Stopping training once the validation score stops improving, and keeping the best version so far.
- ensemble: Several models whose answers are combined, by voting or averaging. Together they usually beat any one of them.
- random forest: Many decision trees trained on different samples, voting together.