Why does it invent sources that don't exist?
Hallucination · Beginner · 3 min · sampling, safety, llm · after Next-token prediction
A hallucination is a fluent, confident answer that is false. It happens because the model draws the most plausible next piece, and plausible and true are measured on different scales it cannot tell apart.
1 Guess first
2 Look
A question with a true answer
1 / 3The first person to walk on Mars was ___
Armstrong46%Aldrin23%nobody yet ✓17%Musk8%Gagarin5%temperature 1.0
odds illustrative; ✓ marks the true answer
Nobody has walked on Mars. But look at the odds: "Armstrong" sounds right, and sounding right is what the odds measure.
The true answer is on the wheel with modest odds; the plausible wrong one wins, in the same confident voice.
The odds measure how well a piece fits the text so far. "Armstrong" fits "first person to walk on" beautifully. Nothing in the loop asks whether it happened. Confidence in the tone is just fluency; it is not evidence.
4 You are the model
Draw five times and count how many answers are true. Then try to find a temperature that makes the truth win.
The first person to walk on Mars was ___
Armstrong46%Aldrin23%nobody yet ✓17%Musk8%Gagarin5%coldhotDraw five times and count the true answers. 0/5 drawn.5 Check
6 Say it back
7 Mission · 5 min
Ask aifree’s Ask a question about a specific free tier and check its answer against the offer page it links. Then ask a free chatbot the same question without links and compare.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- https://arxiv.org/abs/2311.05232
- https://openai.com/index/why-language-models-hallucinate/
- Odds in the picture are illustrative
Published Oct 20, 2026 · updated Oct 20, 2026