Glossary · Phase 4 · Deep learning
Gradient clipping
Capping how big the gradient can get before a step, so one bad batch can't blow up training.
Learn it properly
Vanishing and exploding gradients · Why deep networks were hard to train. About 30 minutes.
Other terms from the same lesson
- vanishing gradients: When the error signal shrinks to almost nothing (or blows up) as it passes back through many layers, so learning stalls.