Infrastructure
Mixed Precision (FP16 / BF16)
Training in lower-precision floats to double throughput and halve memory versus FP32.
Definition
Mixed precision keeps activations and weights in 16-bit floats (FP16 or BF16) while maintaining FP32 master copies for the optimiser. BF16 has the same exponent range as FP32, avoiding the loss-scaling tricks needed for FP16. It is the default training regime today.
Common use cases
- Training
- Inference
- Memory reduction