Why is the mini or flash model the free one?
Knowledge distillation (teacher and student models) · Beginner · 5 min · llm, training · after Next-token prediction, Training: loss and gradient descent, Model size, free vs paid tiers
Distillation trains a small student model to copy a big teacher model: its odds for every next word, not only its answer. The student keeps much of the skill with fewer parameters, so each token costs less to run.
1 Guess first
2 Look
A teacher’s odds, copied
1 / 5The cat sat on the ___
teacher · GPT-2 · 124M parameters · its top 6 next words
floor7.6%bed6.5%couch5.4%ground5.2%edge4.8%bench3.2%real odds: GPT-2 small (teacher) and DistilGPT2 (student) · computed 2026-10-22
The big teacher does not just answer “floor”. It gives every next word odds.
The other way to shrink a model: quantization · find a free small model
A teacher’s odds teach more than its answer, and a student trained to match them ends up close to the teacher with a third fewer parameters, except on rarer facts.
A model’s parameters are the numbers it multiplies for every token it writes. DistilGPT2 has 82 million, GPT-2 124 million: a third less memory and work per token. Google says its smaller Gemini 2.5 models, Flash size and below, are distilled from bigger ones. Cheaper per token is what lets a vendor give the small one away.
4 You are the model
You are the student. Guess the teacher’s next word three times, and compare what each kind of feedback teaches you.
round 1 of 3 · you are the student · guess the teacher’s next word
The cat sat on the ___
real odds: GPT-2 small (teacher) and DistilGPT2 (student) · computed 2026-10-22
Pick the word you think the teacher rates highest.5 Check
The copy is cheaper, not equal: rare facts and hard reasoning slip first. DeepSeek distilled its 671B-parameter R1 into smaller open models. On the AIME 2024 maths test R1 scores 79.8, the 32B student 72.6 and the 1.5B student 28.9. For short, common jobs the free small model is often enough.
7 Say it back
8 Mission · 5 min
In Google AI Studio (free), send the same two prompts to Gemini 3.8 Flash and to the smaller Gemini 3.5 Flash-Lite. First an easy job: “Rewrite politely: send me the report now.” Then a rare fact: “Who won the 1904 Olympic marathon, and what was odd about the race?” Note where the small one keeps up and where it slips.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- https://huggingface.co/distilbert/distilgpt2 (model card, checked 2026-10-22; no update date): DistilGPT2 “has 82 million parameters, was developed using knowledge distillation” with “the supervision of the 124 million parameter version of GPT-2”; WikiText-103 test perplexity 16.3 for GPT-2 and 21.1 for DistilGPT2
- Next-word odds in the picture and the role: GPT-2 small and DistilGPT2 weights from Hugging Face, computed 2026-10-22 with scripts/explain-data/distill.ts (GPT-2 “floor” 0.0764 matches app/lib/explain/data/sampling.json)
- https://arxiv.org/abs/1503.02531 (Hinton, Vinyals, Dean, 2015-03-09): training a small model on the teacher’s “soft targets” (its class probabilities)
- https://blog.google/technology/ai/google-gemini-update-flash-ai-assistant-io-2024/ (2024-05-14, checked 2026-10-22): 1.5 Flash “has been trained by 1.5 Pro through a process called distillation”
- https://arxiv.org/abs/2507.06261 (Gemini 2.5 technical report, v6 2025-12-19, checked 2026-10-22), §2.1: “The smaller models in the Gemini 2.5 series — Flash size and below — use distillation”, approximating “the teacher’s next token prediction distribution” with a k-sparse distribution
- https://huggingface.co/deepseek-ai/DeepSeek-R1 (checked 2026-10-22; no update date): DeepSeek-R1 671B total parameters (37B activated), AIME 2024 pass@1 79.8; distilled from R1 samples: R1-Distill-Qwen-32B 72.6, R1-Distill-Qwen-1.5B 28.9
- https://ai.google.dev/gemini-api/docs/pricing (updated 2026-10-19, checked 2026-10-22): Gemini 3.8 Flash and Gemini 3.5 Flash-Lite “Free of charge” on the free tier; Gemini 3.1 Pro Preview and Gemini 4 Argon “Not available”
- aifree.dev catalog: google-gemini-developer-api-free-tier verified 2026-10-12, models include gemini-3.8-flash and gemini-3.5-flash-lite
- https://arxiv.org/abs/2001.08361 (Kaplan et al., 2020): forward-pass compute per token is about 2 × parameters, the rule of thumb behind “a third less work”
Published Oct 22, 2026 · updated Oct 22, 2026