🧠 AI Memory Calculator

How much memory does a local AI model need – for the model itself and for its context window? Pick a model, a quantization and a context length, and see whether it fits on your graphics card. Everything is calculated in your browser.

Model

Model Parameters [billions] Layers KV heads key-value heads, not all heads Head size Model quantization

Layers, KV heads and head size are in the model's config.json (num_hidden_layers, num_key_value_heads, head_dim or hidden_size / num_attention_heads). The presets are filled in for you.

Context

Context length [tokens] …or your own [tokens] KV cache type Parallel requests each gets its own context My GPU memory [GB] 0 = no GPU, CPU only

Results

Memory by context length

Model weightsKV cache (context)OverheadYour GPU

What takes the memory

The model weights – billions of numbers learned in training. Stored with full precision (16 bits) an 8-billion-parameter model needs 16 GB. Quantization stores them with fewer bits; with Q4_K_M, which Ollama uses by default, the same model needs about 5 GB and is only a little less accurate.

The KV cache – the model's working memory for the context window. For every token it has read, every layer keeps a key and a value vector, so that it doesn't have to read the whole text again for the next word. It grows linearly with the context length, and with long contexts it can become bigger than the model itself.

weights  = parameters × bits / 8
KV cache = 2 × layers × KV heads × head size × bytes × context length × parallel requests
           (2 = one key and one value)

Llama 3.1 8B, F16 cache:  2 × 32 × 8 × 128 × 2 bytes = 131 072 bytes = 128 KiB per token
          8 192 tokens → 1 GiB        131 072 tokens → 16 GiB

Overhead – working buffers for the calculation, here estimated as 0.5 GB plus 5 % of the rest. It depends on the program and the version, so treat the total as an estimate, not a promise.

Tips

If it doesn't fit on the GPU, Ollama puts part of the model in normal RAM and on the CPU – it still works, but many times slower. ollama ps shows the split (for example 48%/52% CPU/GPU) and the context length in use.
Set the context with OLLAMA_CONTEXT_LENGTH=32768 ollama serve, the slider in the Ollama app, or "options": {"num_ctx": 32768} in an API request.
A quantized KV cache (OLLAMA_KV_CACHE_TYPE=q8_0, needs OLLAMA_FLASH_ATTENTION=1) halves the context memory with almost no loss.
Newer models with sliding-window or hybrid attention (Gemma 3, some Qwen and Llama 4 models) need much less KV cache than this formula says – the calculator gives the upper limit for them.
About tokens: in English one token is roughly ¾ of a word; Slovenian and other smaller languages need noticeably more tokens for the same text.

The background – tokens, context, why the model "forgets" and what skills are for – is in my note Tokens and Context: What Happens to Your Message. Setting up Ollama is in Constructing an AI.

An estimate. Real memory use depends on the program, its version, the GPU driver and the exact model file.