Why is it bad at multiplying big numbers?
Arithmetic in LLMs: digits cut into tokens, answers written left to right · Beginner · 5 min · tokens, llm · after Tokens
A language model multiplies by predicting the answer one token at a time, left to right, with no scratch paper. Tokens ignore place value and carries run right to left, so exact answers collapse as digits grow.
1 Guess first
2 Look
How it reads, how it writes, how often it is right
1 / 34739×8216=38935624ones- 1234 →1234“123” here means 1,23_
- 12345 →12345“123” here means 12,3__
Tokens measured with o200k_base (GPT-4o), tiktoken 0.14.0, 2026-10-23
Token edges ignore the columns: the ones digit hides inside “24”, and “123” can mean 1,230 or 12,300.
The answer’s first piece needs every carry, and carries start at the end it hasn’t written yet.
On paper you start at the ones column and carry leftward, writing every partial product down. A model gets one pass per token, writes the biggest digits first, and keeps no working unless it writes it out. So it estimates: the start and end often look right, the middle is a guess.
4 You are the model
You are the model. Write 4739 × 8216 one token at a time, left to right, without any working. Pick each piece.
4739 × 8216 = …
piece 1 of 3 · Write the first piece of 4739 × 8216. No scratch paper.
Pieces as GPT-4o’s tokenizer cuts the answer (o200k_base (GPT-4o), tiktoken 0.14.0, 2026-10-23)
You are the model. Write the answer one piece at a time, left to right, with no working.5 Check
Two fixes. Ask it to work step by step: written partial products become its scratch paper, and reasoning models do this by themselves. Better still, have it run code or a calculator through tool calling: the tool does the carries exactly. For money or science, check any number it did in its head.
7 Say it back
8 Mission · 5 min
In ChatGPT Free, send “Reply with only the number: 48271 × 93516”. Check it with your phone’s calculator. Then in a new chat send “Work out 48271 × 93516 step by step, or use code.” Which answer matched, and which digits were wrong the first time?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Get an email when new lessons come out
New lessons arrive in the weekly digest, only in a week that has one. You confirm by email first; unsubscribe in one click.
Go deeper
Sources
- Tokens measured with OpenAI’s tiktoken 0.14.0 on 2026-10-23: o200k_base (GPT-4o) cuts “4739 × 8216 = 38935624” into 473|9 × 821|6 = 389|356|24, “1234” into 123|4 and “12345” into 123|45; gpt2 cuts 4739 into 47|39 and 8216 into 82|16. 4739 × 8216 = 38935624 and 39 × 16 = 624 (exact arithmetic)
- Dziri et al., “Faith and Fate: Limits of Transformers on Compositionality” (arXiv:2305.18654, v1 2023-05-29, v3 2023-10-31, https://arxiv.org/abs/2305.18654, checked 2026-10-23). Figure 16 (App. B.2), GPT-4 few-shot with question-answer examples, exact-answer accuracy on n-digit × n-digit: 1×1 100%, 2×2 98%, 3×3 56%, 4×4 3%, 5×5 0%; 500 test examples per size. §1: zero-shot 3-digit × 3-digit, GPT-4 59%, ChatGPT 55%
- Same paper §3.2.1: the first digit(s) of the output correlate with the first digit(s) of each input and the last digit(s) depend only on the last digit(s) of each input; models learn these surface patterns. App. B.5: few-shot scratchpad prompting raises GPT-4’s accuracy but it still falls toward zero as size grows
- Anand S., “Are LLMs any good at mental math?” (s-anand.net, 2025-04-27, https://www.s-anand.net/blog/are-llms-any-good-at-mental-math/, checked 2026-10-23): informal test of ~50 LLMs on 7 multiplications; no model got 987,654,321 × 123,456,789 right; OpenAI reasoning models (o1-mini, o3, o4-mini…) got 6 of 7
- The wrong options in the you-are-the-model step (385, 536, 54…) are illustrative distractors; 385 comes from the estimate 4,700 × 8,200 ≈ 38.5 million
- aifree.dev catalog, chatgpt-free (verified 2026-10-12): free text chats on chatgpt.com
- The mission’s 48271 × 93516 is an example pair of 5-digit numbers; the exact product is 4,514,110,836 (Python integer arithmetic, 2026-10-26). In the token question “123” stands for 1,230 in 1234 (123|4) and 12,300 in 12345 (123|45), place value of the o200k_base pieces measured above
Published Oct 23, 2026 · updated Oct 26, 2026