Why does a 7B model need 16 GB of RAM?
Model memory: parameters × bytes, plus the KV cache · Beginner · 5 min · llm, context · after Open weights vs closed API models, and model licenses, Quantization (bits per weight), Context window
A model must hold every weight in memory: parameters × bytes per parameter, plus a KV cache that grows with the context. A 7B model needs about 14 GB at 16 bits and 4–5 GB at 4 bits.
1 Guess first
2 Look
Where the gigabytes go
1 / 5Qwen2.5 7B: 7,615,616,512 parameters
× 2 bytes each (16-bit)
= 15.2 GB, before you type a wordThe real download, qwen2.5:7b-instruct-fp16: 15 GB. A 16 GB card is already full.
Hugging Face model pages, Ollama library, NVIDIA, Hugging Face and Cloudflare docs · checked 2026-10-22
7.6 billion numbers, 2 bytes each: 15 GB of memory before the chat even starts.
Fewer bits per weight: quantization · what the cache holds: context window · free GPU quotas · run one locally
Memory is parameters × bytes per parameter, plus a cache that grows with the context: “7B” at 16 bits is 14 GB before you type, and 4 bits cuts it to about 4–5 GB.
A parameter is one learned number. At 16 bits it takes 2 bytes, at 8 bits one, at 4 bits about half. So the weights need parameters × bytes: 7B × 2 = 14 GB. On top sits the KV cache: for every token in the chat, each layer keeps a key and a value. Longer context, more memory.
4 You are the model
Four models, four free places. For each, add weights, cache and runtime, and pick the place it fits.
model 1 of 4 · pick the free place it fits
Mistral 7B at 4-bit (4.4 GB) for short chats, 4,096 tokens.
Hugging Face model pages, Ollama library, NVIDIA, Hugging Face and Cloudflare docs · checked 2026-10-22 · runtime share illustrative
Add it up: weights + cache + about 1 GB. Where does it fit?5 Check
VRAM is the graphics card’s own memory, the fast kind. When a model is bigger than one card, Ollama spreads it over every GPU, then into normal RAM: `ollama ps` shows a split like “48%/52% CPU/GPU”. It still answers, only slower. Too big for your RAM as well? Use a hosted free API.
7 Say it back
8 Mission · 5 min
In a free Kaggle notebook (phone-verified), set Accelerator to “GPU T4 x2” and run !nvidia-smi. Read each card’s total memory. Then do the sum: does Qwen2.5 7B at 16-bit (15 GB) plus a 4,096-token cache fit one card? Does the 4-bit file (4.7 GB)?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- Hugging Face model API, safetensors parameter totals (checked 2026-10-22): meta-llama/Llama-3.1-8B-Instruct 8,030,261,248; Qwen/Qwen2.5-7B-Instruct 7,615,616,512; mistralai/Mistral-7B-Instruct-v0.3 7,248,023,552; google/gemma-3-27b-it 27,432,406,640 (all BF16)
- https://ollama.com/library/llama3.1/tags, /qwen2.5/tags, /mistral/tags, /gemma3/tags (checked 2026-10-22): llama3.1:8b-instruct fp16 16GB, q8_0 8.5GB, q4_K_M 4.9GB; qwen2.5:7b-instruct fp16 15GB, q8_0 8.1GB, q4_K_M 4.7GB; mistral:7b-instruct-v0.3 fp16 14GB, q8_0 7.7GB, q4_K_M 4.4GB; gemma3:27b-it fp16 55GB, q8_0 30GB, q4_K_M 17GB
- https://ollama.com/library/llama3.3 (checked 2026-10-22): llama3.3:70b 43GB (default 4-bit tag)
- https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3/blob/main/config.json (checked 2026-10-22): num_hidden_layers 32, num_key_value_heads 8, hidden_size 4096, num_attention_heads 32 → head size 128; KV cache at 16 bits = 2 × 32 × 8 × 128 × 2 bytes = 131,072 bytes (128 KiB) per token; 4,096 tokens ≈ 0.54 GB, 32,768 ≈ 4.3 GB (arithmetic)
- https://docs.ollama.com/faq (checked 2026-10-22): “By default, Ollama uses a context window size of 4096 tokens”; ollama ps Processor column, “48%/52% CPU/GPU means the model was loaded partially onto both the GPU and into system memory”; “If the model does not fit entirely on one GPU, then it will be spread across all the available GPUs”
- https://www.nvidia.com/en-us/data-center/tesla-t4/ (checked 2026-10-22): GPU memory 16 GB GDDR6
- aifree.dev catalog, kaggle-free-gpu-tpu (verified 2026-10-12): up to 30 hours/week GPU; “Accelerators: NVIDIA Tesla P100 (16GB), 2x NVIDIA T4, or TPU v3-8”; phone verification required
- https://huggingface.co/docs/hub/spaces-zerogpu (checked 2026-10-22): default size large = half NVIDIA RTX Pro 6000 Blackwell, 48GB; free account 5 minutes daily GPU quota; Gradio SDK only
- https://developers.cloudflare.com/workers-ai/platform/pricing/ (checked 2026-10-22): 10,000 Neurons per day free; @cf/meta/llama-3.3-70b-instruct-fp8-fast listed
- https://research.google.com/colaboratory/faq.html (checked 2026-10-22): “The types of GPUs and TPUs that are available in Colab vary over time”; “Colab does not publish these limits”
- The 1 GB runtime share in the picture and the role is illustrative, labelled as such; “slower” on CPU is qualitative, not measured here
Published Oct 22, 2026 · updated Oct 22, 2026