Infrastructure
FP4
4-bit floating-point format introduced on Blackwell GPUs — double FP8 throughput at inference.
Definition
FP4 packs an exponent and minimal mantissa into 4 bits, doubling the throughput of FP8 on NVIDIA's Blackwell architecture (B200). Combined with second-gen Transformer Engine it enables inference at extreme speeds with managed quality loss.
Common use cases
- Blackwell inference
- Large-MoE serving
- Cost reduction