Double descent

When a model is made far bigger than the data seems to justify, its test error can fall again after first rising. It's one reason huge neural networks still generalize.

Other terms from the same lesson

• bias-variance tradeoff: Simple models miss in a steady way (bias); flexible models swing with every change in the data (variance). The best model balances the two.
• learning curve: A chart of training and validation scores as the training set grows. The gap between the lines shows whether more data would help.