Does it plan what it's going to say?
Next-token prediction · Beginner · 3 min · sampling, llm · after Tokens
A language model writes one piece at a time: it produces odds for the next piece, draws one, appends it, and starts over. There is no plan for the sentence, only the next piece, every time.
1 Guess first
2 Look
One piece at a time
1 / 2The cat sat on the ___
mat60%sofa20%floor16%moon3%temperature 1.0
odds illustrative
Draw. The piece lands at the end of the sentence, and the whole trip runs again for the next one — with new odds.
Every Draw appends a piece and the odds for the next one change: the sentence is built, not retrieved.
Loop: read everything so far → odds for the next piece → draw one → append it → repeat until a stop piece comes up. Long, coherent answers come from the odds being very good, not from a plan.
4 You are the model
You go first: pick the next piece. Then Draw to see what the wheel rolled. Three rounds.
The cat sat on the ___
mat60%sofa20%floor16%moon3%temperature 1.0
You first: pick the next piece.5 Check
6 Say it back
7 Mission · 2 min
Give a free chatbot half a sentence — "The best thing about living in Hanoi is" — and press send five times, starting fresh each time. Count how many different endings you get.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- https://arxiv.org/abs/2005.14165 (GPT-3 paper, autoregressive generation)
- Odds in the picture are illustrative until E2b (#220)
Published Oct 20, 2026 · updated Oct 20, 2026