Why does it get worse the longer we chat?
Long chats: a model guesses early, builds on its own guesses turn after turn, and gets less reliable as the context grows, even well inside the window · Beginner · 5 min · context, llm · after Lost in the middle: position bias in long contexts
Long chats get worse: a model guesses early, then builds on its own guesses, and answers get shakier as the context grows, even inside the window. A new chat with a short summary and the whole request helps.
1 Guess first
2 Look
What a long chat costs
1 / 4Average score, same tasks, same details
one message≈90turn by turn≈65the same pieces pasted into one message: 95.1% of the one-message score
15 LLMs, 600 tasks, 200,000+ simulated chats (Laban et al., 2025)
The same tasks scored about 90 in one message and about 65 when the details came turn by turn. Averaged over models and tasks: a 39% drop.
A chat goes wrong when the model answers before it has everything and then keeps building on that guess, and a longer context makes every answer a little shakier.
Why the chat itself hurts: a model answers every turn, even when half the request is still missing. It fills the gaps with guesses, and those guesses stay in the context. Next turn it reads its own earlier answer as if it were fact and patches it instead of starting clean. Early wrong guesses ride along to the end.
4 You are the model
You are the model. A user plans a trip with you, one detail at a time, then starts a new chat. Pick what you would most likely write each turn.
round 1 of 3 · Turn 1. The user writes: “Plan a 3-day trip for me.” Nothing else yet.
What do you most likely write?
15 LLMs, 600 tasks, 200,000+ simulated chats (Laban et al., 2025); replies illustrative
You are the model. Pick what you would most likely write.5 Check
Length hurts on its own too. In one test, GPT-4o found a hidden fact 99.3% of the time in a short text but 69.7% of the time in 32K tokens, a quarter of its window. This is not about where the fact sits, like lost in the middle: more text makes every answer shakier.
So when a chat starts going in circles, stop patching it. Ask for a short summary of everything you want, open a new chat, and paste the summary with your full request in one message. Next time, put the whole request up front instead of feeding it in piece by piece.
8 Say it back
9 Mission · 5 min
In ChatGPT, send “Plan a 3-day trip for me.” Then add one detail per message: “It’s Hanoi.”, “We travel with a toddler.”, “No museums.”, “About $50 a day.” Send “Summarize everything I want in one message.”, open a new chat, paste that summary and ask for the plan. Which plan fits all four details?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Get an email when new lessons come out
New lessons arrive in the weekly digest, only in a week that has one. You confirm by email first; unsubscribe in one click.
Go deeper
Sources
- Laban, Hayashi, Zhou, Neville, “LLMs Get Lost In Multi-Turn Conversation” (arXiv:2505.06120, v1 2025-05-09, https://arxiv.org/abs/2505.06120, checked 2026-10-23): 15 LLMs, six generation tasks, 600 instructions, 200,000+ simulated conversations; “an average drop of 39% across six generation tasks”, about 90% fully specified in one turn vs about 65% sharded over turns; aptitude −16%, unreliability +112%; Concat (the shards in one message) 95.1% of Full; first answer attempt in the first 20% of turns P̄ = 30.9 vs 64.4 in the last 20% (Code and Math); final answers 20–300% longer than single-turn ones; GPT-4o Full 93.0, Concat 90.9, Sharded 59.1, Recap 76.6, Snowball 65.3; at temperature 0.0 unreliability stays around 30%; advice: start a new conversation that repeats the information, after asking for a summary
- Modarressi et al., “NoLiMa: Long-Context Evaluation Beyond Literal Matching” (arXiv:2502.05167, v3 2025-07-09, https://arxiv.org/abs/2502.05167, checked 2026-10-23): 13 LLMs that claim at least 128K tokens; “At 32K, for instance, 11 models drop below 50% of their strong short-length baselines”; GPT-4o “from an almost-perfect baseline of 99.3% to 69.7%”
- Hong, Troynikov, Huber, “Context Rot: How Increasing Input Tokens Impacts LLM Performance” (Chroma, 2025-07-14, https://www.trychroma.com/research/context-rot, checked 2026-10-23): 18 LLMs incl. GPT-4.1, Claude 4, Gemini 2.5, Qwen3; repeated-words task 25 to 10,000 words, “As context length increases, performance consistently degrades across all models”; LongMemEval focused prompts (~300 tokens) vs full (~113k tokens): “significantly higher performance on focused prompts”
- The trip chat, the role’s replies and plans, the summary card, the coding chat and the friend’s project are illustrative, not measured outputs
- aifree.dev catalog, chatgpt-free (verified 2026-10-12, read-only API check 2026-10-23): ChatGPT Free Plan
Published Oct 23, 2026 · updated Oct 23, 2026