🖥️ GGUF / AWQ / EXL2 • KV Cache Math • Memory Bandwidth Tokens/s
Local LLM GPU VRAM & Speed Calculator
Calculate exact GPU memory footprint (Model Weights + KV Cache + CUDA Buffers) and estimate token generation speed across RTX 4090, RTX 5090, Mac Studio, and H100.
⚙️ Model & Quantization Setup
Context Window (KV Cache) 16,384 tokens
Total Required GPU VRAM
Fits 24GB GPU (RTX 3090 / 4090)
21.4 GB
MODEL WEIGHTS 19.2 GB
KV CACHE 1.4 GB
CUDA OVERHEAD 0.8 GB