Size a GPU box before you rent it.

Pick a model, a weight quantization, a card and a context length. The calculator works out the KV cache each agent session needs and how many sessions fit in the VRAM that is left.

Model

Weights

Card VRAM (GB)

Context per agent

KV cache quant

How the card's memory is spent
Measure	Value
Agents that fit	-
KV cache per agent	-
Fixed: weights plus buffers	-
Used of card	-

Why parameter count is the wrong number to look at

KV cache per token is 2 × full_attention_layers × kv_heads × head_dim × bytes. Only layers with full attention carry a cache that grows with context. Sliding-window layers (gpt-oss alternates them with full-attention layers) and linear-attention layers in hybrid models hold a small, constant state instead. So the number that matters is full-attention layers × KV heads × head dim, not parameter count.

Model	Full-attn layers	KV heads	Head dim	Weights (Q4_K_M / MXFP4)	GiB per 131k slot (q8_0 KV)
Qwen3-32B	64 of 64	8	128	18.4 GiB	16.0
Qwen3-30B-A3B	48 of 48	4	128	17.3 GiB	6.0
Qwen3-8B	36 of 36	8	128	4.7 GiB	9.0
gpt-oss-20b	12 of 24	8	64	11.3 GiB	1.5

Qwen3-8B has about a quarter of the weights of Qwen3-30B-A3B but needs half again as much cache per session, because it keeps twice the KV heads. Past a handful of concurrent long-context sessions the smaller model is the more expensive one to serve.

Things that catch people out

Weights are the cheap part once cache dominates. With many long-context sessions on a large card, moving from Q4_K_M to Q8_0 weights costs a fixed amount of VRAM and only a little decode speed, because every step is bound by reading KV, not weights.
Context beyond the trained window needs RoPE scaling. Qwen3's config sets max_position_embeddings to 40,960; running 65k or 131k sessions requires a scaling method such as YaRN enabled in your inference server.
A reasoning model returns empty content at low max_tokens. The budget goes inside <think> first. Give it 300-400+ tokens and read reasoning_content before concluding the server is broken.
Verify the arithmetic against a real run. Expect measured usage to land a GB or two above the prediction; the gap is compute buffers and allocator overhead.

Method

Architecture numbers (layers, KV heads, head dim, full vs sliding attention) come from each model's public config.json on Hugging Face: Qwen/Qwen3-32B, Qwen/Qwen3-30B-A3B, Qwen/Qwen3-8B and openai/gpt-oss-20b. Weight sizes are the GGUF file sizes in Qwen/Qwen3-32B-GGUF, Qwen/Qwen3-30B-A3B-GGUF, Qwen/Qwen3-8B-GGUF and ggml-org/gpt-oss-20b-GGUF. The calculator adds a flat 2 GiB for compute buffers, treats 96% of the card as usable, and counts q8_0 as 1 byte per cache element. Sliding-window cache in gpt-oss (128 tokens per layer) is small enough to ignore.

Estimates, not guarantees: real usage depends on the inference server, batch settings and driver.