LLM VRAM Calculator
Will this model fit your GPU? Pick a model, a quantization, your context length and your card, get an instant verdict, the exact weights / KV-cache / overhead breakdown, and a rough tokens-per-second estimate. No sign-up, no black box: the formula is shown right on the page and the whole thing runs in your browser.
Pick your setup
- Weights - GB
- KV cache - GB (approx)
- Overhead - GB
Generation speed once loaded. Time-to-first-token depends on prompt length and is separate.
Show the exact formula used
These approximate llama.cpp / GGUF memory behaviour. Values below plug in your current selection.
weights = params × bits ÷ 8 = -
KV cache = -
overhead = min(0.6 + 0.06 × params, 2.5) = -
total = weights + KV + overhead = -
It fits, now decide if you want the project or the payoff
Your card can handle this, and the guides below will get it running. But if what you actually wanted was an uncensored companion rather than a weekend of GGUF wrangling, Ember is that stack finished: one installer, her own tuned brain, voice and memory, running on this same card. $29 paid once in crypto, no account, yours forever. 18+.
See Ember →Too big for this card? You've got options
Drop to a smaller quant or a smaller model, shorten the context, or step up to a bigger GPU. And if the thing you wanted the VRAM for is an uncensored companion, note that Ember ships a 9B brain sized for an 8 GB NVIDIA card, so she can fit where the model above does not. $29 paid once in crypto, yours forever. 18+.
See what Ember needs →See the best GPUs for local LLMs by budget and the full hardware guide.
How to read the result
Three things decide whether a local model fits: the weights (fixed by the parameter count and the quant you pick), the KV cache (grows with context length and shrinks if you quantize it), and a bit of runtime overhead. Add them up and compare to your card's VRAM. If you're under ~90% you're comfortable; up to 100% is tight but workable; over that, NVIDIA and AMD cards offload the rest to system RAM (slow), while Apple Silicon simply can't, unified memory has no overflow.
The single biggest lever is quantization. Dropping from FP16 to Q4_K_M roughly quarters the weight size for a barely-perceptible quality hit, which is why it's the default here and the sweet spot for local use. For the full trade-off ladder see the GGUF quantization cheat sheet. If you're shopping by capacity, the best local LLMs for 24GB and the VRAM needed for a 70B model are the two guides people hit most. Confused about system RAM versus VRAM? Read RAM vs VRAM for local AI.
A note on Mixture-of-Experts models (Mixtral, gpt-oss): every expert has to sit in VRAM, so they cost the full parameter count to load, but only the active experts are read per token, so they generate far faster than a dense model of the same size. The calculator accounts for both: total params drive the VRAM, active params drive the tokens/sec.
VRAM calculator FAQ
How much VRAM do I need to run an LLM?
As a rule of thumb, VRAM needed ≈ (parameters in billions × bits-per-weight ÷ 8) for the weights, plus the KV cache (grows with context length) plus ~0.6-2.5 GB of overhead. A 7-8B model at Q4_K_M needs roughly 5-6 GB; a 70B model at Q4_K_M needs about 40-45 GB. This calculator does the exact math for your specific model, quant, context and GPU.
What is the KV cache and why does it use VRAM?
The KV cache stores the attention keys and values for every token already in the context so the model doesn't recompute them each step. Its size is 2 × layers × KV-heads × head-dim × context-length × bytes-per-element. Longer context and higher KV precision (FP16 vs Q8/Q4) both increase it, for long-context runs the KV cache can rival the weights in size.
Does a MoE model like Mixtral or gpt-oss need less VRAM?
No. A Mixture-of-Experts model must hold ALL of its parameters in VRAM (every expert can be routed to), so a 46.7B MoE takes ~46.7B of weight VRAM. What MoE saves is speed: only the active parameters (e.g. ~12.9B for Mixtral) are read per token, so tokens/sec is much faster than a dense model of the same total size.
What happens if the model doesn't fit in VRAM?
On NVIDIA/AMD you can offload the overflow layers to system RAM, it still runs, just much slower because those layers stream over the PCIe bus. On Apple Silicon and unified-memory APUs there's no separate VRAM to overflow into, so exceeding the working-set budget is a hard fail. If it won't fit, either drop to a smaller quant, shorten context, or step up to a bigger GPU.
How accurate are these numbers?
The weights and (for listed models) the KV-cache math are exact for the given architecture and closely track real llama.cpp/GGUF memory use. Overhead and tokens/sec are deliberately labelled rough, real throughput depends on your backend, batch size, flash-attention, and prompt length. Treat the verdict as a reliable fit check and the speed as a ballpark.
Estimates approximate llama.cpp/GGUF memory use and typical throughput. Real numbers vary with backend, flash-attention, batch size and prompt length. We link to tools we make, no ads, no trackers.