Glossary · Phase 4 · Deep learning
FLOPs
Floating-point operations: the count of basic multiply-and-add steps. Training cost and chip speed are both measured in them.
Learn it properly
GPUs and compute · Why deep learning runs on graphics cards. About 30 minutes.
Other terms from the same lesson
- bf16: A 16-bit number format with the same range as normal 32-bit numbers but less precision. It halves memory and speeds up training.
- GPU: A graphics chip that does thousands of multiplications at once, which is why it trains neural networks fast.
- mixed precision: Doing most of the math in smaller, faster number formats like bf16, while keeping a full-precision copy where accuracy matters.