Glossary · Phase 4 · Deep learning
Vanishing gradients
When the error signal shrinks to almost nothing (or blows up) as it passes back through many layers, so learning stalls.
Learn it properly
Vanishing and exploding gradients · Why deep networks were hard to train. About 30 minutes.
Other terms from the same lesson
- gradient clipping: Capping how big the gradient can get before a step, so one bad batch can't blow up training.