How does a 30 GB model fit on my laptop?
Quantization (bits per weight) · Builder · 5 min · llm, transformer · after Training: loss and gradient descent
Quantization stores each model weight with fewer bits, for example 4 instead of 16, so the file and memory shrink about four times. Quality barely moves down to 4 bits and collapses below that; the context needs memory on top.
1 Guess first
2 Look
Same model, fewer bits
1 / 4Gemma 3 27B: one model, three downloads
16-bit55 GB8-bit30 GB4-bit17 GBdownload sizes: ollama.com/library/gemma3/tags (checked 2026-10-21)
One model, three downloads: 55 GB, 30 GB, 17 GB. Same weights, stored with fewer bits.
File size is weights × bits: 16 → 4 bits shrinks a model about four times, costs a few points at 4 bits, collapses at 3, and the context needs memory on top.
A weight is one learned number, normally stored in 16 bits. Quantization rounds each weight to fewer bits: 8 is almost lossless, 4 costs a little. The rounding is done once, before you download. Tags like `q8_0` or `q4_K_M` on Ollama name the scheme; `fp16` is the unrounded file.
4 You are the model
You choose the download. For each laptop, pick the most capable Gemma 3 file that still leaves room: more parameters first, then more bits.
laptop 1 of 3 · keep the download under 75 % of RAM
A laptop with 8 GB of RAM
sizes: ollama.com/library/gemma3/tags (checked 2026-10-21) · the 75 % rule is an illustrative rule of thumb
Pick the most capable download that still fits: more parameters first, then more bits.5 Check
6 Say it back
7 Mission · 5 min
Install Ollama (free), then run: ollama pull gemma3:270m and ollama pull gemma3:270m-it-fp16 (292 MB and 543 MB). Compare their sizes with ollama ls, ask both the same question with ollama run, and check memory with ollama ps. Then try your laptop’s best fit from the role.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- https://ollama.com/library/gemma3/tags (checked 2026-10-21): 27b-it-fp16 55GB, 27b-it-q8_0 30GB, 27b q4_K_M 17GB; 12b 24/13/8.2GB; 4b 8.6/5.0/3.4GB; 1b 2.0/1.1GB; 270m (q8_0) 292MB, 270m-it-fp16 543MB
- https://developers.googleblog.com/en/gemma-3-quantized-aware-trained-state-of-the-art-ai-to-consumer-gpus/ (2025-04-18, checked 2026-10-21): weights-only memory BF16 → int4: 27B 54 → 14.1 GB, 12B 24 → 6.6, 4B 8 → 2.6, 1B 2 → 0.5; “Running the model also requires additional VRAM for the KV cache”
- https://arxiv.org/abs/2505.02214 (Zheng et al., 2025-05-04; checked 2026-10-21): Table 4, Qwen3-8B MMLU, per-group (128) PTQ: FP16 74.7; AWQ 8-bit 74.5, 4-bit 69.3, 3-bit 33.2, 2-bit 25.3; GPTQ 4-bit 73.4
- https://docs.ollama.com/cli (checked 2026-10-21): ollama pull, ollama ls, ollama run, ollama ps
- The 75 % fit rule in the role and the weights/cache split in the picture are illustrative, labelled as such
Published Oct 21, 2026 · updated Oct 21, 2026