Why does it ‘think’ before answering?
Reasoning (thinking) models · Builder · 5 min · llm, tokens · after Next-token prediction
A reasoning model writes extra tokens, usually hidden, working through a problem before its answer. Those steps help on multi-step problems, add cost and wait time, and cannot supply facts the model never learned.
1 Guess first
2 Look
Answer now, or think first?
1 / 4"A bat and a ball cost $1.10. The bat costs $1.00 more than the ball. How much is the ball?"
The answer that jumps out:10¢ ✗the classic first-response trap (Cognitive Reflection Test, Frederick 2005)
The first answer that comes to mind is 10¢. It is wrong: the bat would then cost $1.10.
Written steps catch the trap answer; you pay for them as output tokens; and the same model gains a lot on maths and code but barely on knowledge or chat.
A model only produces the next token, so the only way to “think longer” is to write more. That is called test-time compute: spending more tokens when answering. Vendors bill those thinking tokens as output, and they make you wait. Most apps hide them or show a short summary instead.
4 You are the model
You decide where thinking is worth its tokens. For each kind of task, choose Think first or Answer now, then see what the same model scored both ways.
task 1 of 6 · is thinking worth the tokens here?
Maths contest (AIME’24)
Qwen3-235B-A22B, thinking off vs on · Qwen3 Technical Report, Tables 11–12 (2025)
Spend the tokens (Think first) or save them (Answer now)? Then see what the same model scored both ways.5 Check
6 Say it back
7 Mission · 5 min
In Google AI Studio (free), pick Gemini 3.8 Flash and ask: “If today is Wednesday, what day is it in 100 days?” Open its thoughts. Then ask “What is the capital of Japan?” Compare how long each one thinks and how much it writes.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Go deeper
Sources
- https://ai.google.dev/gemini-api/docs/thinking (updated 2026-10-14, checked 2026-10-21): thinking tokens billed as output, thought summaries only, low thinking for fact retrieval
- https://ai.google.dev/gemini-api/docs/pricing (updated 2026-10-19, checked 2026-10-21): Gemini 3.8 Flash free of charge on the free tier; output price includes thinking tokens
- https://developers.openai.com/api/docs/guides/reasoning (checked 2026-10-21): reasoning tokens not visible via the API, occupy context, billed as output
- https://developers.openai.com/api/docs/guides/reasoning (checked 2026-10-21): usage example output_tokens 1186 with reasoning_tokens 1024 — the bill in the picture
- https://arxiv.org/abs/2505.09388 (Qwen3 Technical Report, 2025-05-14; checked 2026-10-21): Qwen3-235B-A22B, Table 11 thinking vs Table 12 non-thinking — AIME’24 85.7/40.1, LiveCodeBench v5 70.7/35.3, GPQA-Diamond 71.1/62.9, MMLU-Redux 92.7/89.2, IFEval 83.4/83.2, Arena-Hard 95.6/96.1
- https://doi.org/10.1257/089533005775196732 (Frederick 2005, Cognitive Reflection Test): the bat-and-ball first-response trap
- https://arxiv.org/abs/2201.11903
- The scratchpad in the picture is a worked example written by us, labelled as such, not a model output
Published Oct 21, 2026 · updated Oct 21, 2026