Estimate LLM GPU VRAM in Python Before Loading a Model: Separate Weight Precision, GQA KV Cache, and Runtime Overhead
Multiplying parameter count by bytes per weight is not enough to know whether an LLM will fit on your GPU under real traffic. Before downloading checkpoints or launching a serving container, calculate three separate budgets in binary GiB: effective weight memory including quantization metadata, per-request GQA KV-cache memory across your target context length and concurrency, and runtime activation workspace. Environment and verification scope: Verified November 18, 2026 KST on Windows 11 with Python 3.13 ( 14/14 unit tests passed in execution-evidence.json ) against vLLM PagedAttention memory allocation rules and Hugging Face Transformers Grouped-Query Attention (GQA) configuration schemas. All memory figures use binary gibibytes ( 1 GiB = 2^30 bytes ) to match nvidia-smi and PyTorch device reporting. Procedure and equations covered in this guide 1. The four components of serving VRAM: why weight-only estimates fail 2. Exact equations for effective weights and GQA/MQA KV-cac...