FP8 Quantization & KV Caching on NVIDIA Hopper: vLLM & TensorRT-LLM Serving Architecture

1. The Hopper FP8 Tensor Core Revolution

Prior to the NVIDIA Hopper architecture (H100/H200), INT8 quantization required integer arithmetic pipelines with costly dequantization steps that often introduced non-linear accuracy loss. Hopper introduced native hardware FP8 Tensor Cores providing:

• 2x Compute Density: 1,979 FP8 TFLOPS compared to 989 FP16/BF16 TFLOPS on an H100 SXM5.
• 50% Memory Bandwidth Compression: Weight matrix loads and KV cache attention lookups transfer 1 byte per element instead of 2 bytes, cutting memory bus transit in half.
• Native Exponent Representation: Floating-point dynamic range handles activation distributions far better than linear integer grids (INT8).

2. Numerical Formats: E4M3 vs. E5M2

3. Production vLLM Launch Configuration

Serve pre-quantized FP8 models (e.g. neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8) while enabling FP8 KV caching:

python3 -m vllm.entrypoints.openai.api_server \
  --model neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 \
  --kv-cache-dtype fp8_e4m3 \
  --calculate-kv-scales \
  --tensor-parallel-size 4 \
  --gpu-memory-utilization 0.92 \
  --max-model-len 16384

4. Interactive FP8 Precision Calibrator

Calculate your exact memory savings, VRAM footprints, and batch concurrency boosts using our dedicated tool: