Why does it forget what I said 20 messages ago?
Context window · Beginner · 3 min · context, tokens, llm · after Tokens
The context window is the fixed number of pieces a model can read at once. Everything it answers from must fit inside it; anything older slides out and was simply never read this turn.
1 Guess first
2 Look
The belt and the frame
1 / 31my name is Minh2I live in Hanoi3I like pho4I have a catwindow: the last 6 messages · dim = fallen off the belt
Four messages, a window of six. Everything you said is in front of the model.
The frame stays the same size while the belt keeps moving; message 1 is not faded, it is outside.
Each turn, the model reads the last N pieces — the window — and nothing else. Apps hide this by resending the conversation each time, so "memory" is really "still inside the window". Windows are big now, but every window has an edge.
4 You are the model
Set the window size, add messages, then say whether the model can still see "I have a cat" (message 4). Reveal to check.
1my name is Minh2I live in Hanoi3I like pho4I have a cat5she is greywindow: the last 5 messages · dim = fallen off the belt
tinyroomy5 messages so far. Add more, then: can the model still see message 4 ("I have a cat")?5 Check
6 Say it back
7 Mission · 5 min
With a free Gemini key, paste a long document and ask about the first paragraph. Then paste the same document twice over and ask again. Notice when the answer starts to drift.
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Sources
- https://docs.anthropic.com/en/docs/build-with-claude/context-windows
- https://ai.google.dev/gemini-api/docs/long-context
Published Oct 20, 2026 · updated Oct 20, 2026