Why does it make illegal moves in chess?
Chess: a chatbot predicts the next move as text from the list of moves so far and keeps no real board, so familiar openings go fine but deep in a game it loses track of where the pieces are; giving it the board, a list of legal moves or a chess engine fixes most of it · Beginner · 5 min · context, llm · after Long chats: a model guesses early, builds on its own guesses turn after turn, and gets less reliable as the context grows, even well inside the window, Arithmetic in LLMs: digits cut into tokens, answers written left to right
A chatbot plays chess by predicting the next move as text from the move list; it keeps no real board. Openings go fine, but deep in a game it loses track. Give it the board, or use an engine.
1 Guess first
2 Look
The board you see, the text it gets
1 / 4What you see
What the model gets
1. e4 e5 2. Nf3 Nc6 3. Bb5
No board, no pieces: only text, and the next move is more text.
opening position, a common line
You see a board. The chatbot gets a line of text and predicts the move most likely to come next. This opening appears in millions of game records, so the likely move is also a legal one.
You see a board; the model sees a line of text. Early on that text is familiar, later it is new, and the board the model needs is never written down unless you give it.
A chatbot reads a game the way it reads anything: as text, 1. e4 e5 2. Nf3 Nc6…, and it predicts the move that most likely comes next. There is no board inside, only patterns from the game records it was trained on. A move can look right as text and still be impossible on the board.
4 You are the model
You are the chatbot. You get chess moves as text, never a picture: a famous opening, then a long game that ends in check, then the same position with the board written out.
round 1 of 3 · A new game. You only get the moves as text.
1. e4 e5 2. Nf3 Nc6 3. Bb5
What does Black most likely play next?
positions illustrative; 100% from ChessArena Table 3
You are the chatbot. You get text, never a board.5 Check
Long games make it worse. Every move must be replayed in the head to know where the pieces are, and the list keeps growing. The most common slip Game Arena saw: the model is in check but proposes a move that leaves its king attacked, often a move that is common in game records.
The fix is to stop asking it to remember the board. Write the position out each turn as FEN, a one-line board description, or give the list of legal moves: in ChessArena, DeepSeek-R1’s legal moves went from 44.6% to 82.5% with the board and to 100% with the list. Or let a chess engine pick the moves.
8 Say it back
9 Mission · 5 min
In ChatGPT Free, play from text only: write “Let’s play chess. I’m White. Reply with one move only. 1. e4” and keep going. Enter every move on the free Lichess analysis board, which refuses illegal ones. When ChatGPT plays an illegal move, copy the FEN from Lichess into the chat and ask again. How many moves did it last?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Get an email when new lessons come out
New lessons arrive in the weekly digest, only in a week that has one. You confirm by email first; unsubscribe in one click.
Go deeper
Sources
- Liu et al., “ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models” (arXiv:2509.24239 v4, 2026-04-23, https://arxiv.org/abs/2509.24239, checked 2026-10-24): Blindfold mode gives “the move history from both players in the form of a multi-turn conversation” and the model “must reconstruct the chessboard state internally”; Standard mode gives the board as FEN. Table 3, legal rate (LR) of move selection: DeepSeek-R1 Blindfold 44.6% without / 94.7% with legal moves, Standard 82.5% without / 100.0% with; o3 Blindfold 86.9% / 98.4%, Standard 98.0% / 99.6%. Table 18, average conversation turns, Blindfold without legal moves, successful / failed: DeepSeek-R1 72 / 112, Doubao-Seed-1.6-Thinking 70 / 112, o3 88 / 129, GPT-4.1 94 / 94; “longer conversations impede board reconstruction”
- Doerschuk-Tiberi et al., “Game Arena: Strategic LLM Evaluation in Competitive Environments” (arXiv:2609.31473 v1, 2026-09-25, https://arxiv.org/abs/2609.31473, checked 2026-10-24): each turn the model “receives the current position in Forsyth-Edwards Notation (FEN) and a complete move history in PGN format”, with no list of legal moves; three retries, then the game is lost; best Gemini model “< 0.2” rethinks per game, DeepSeek V3.2 and Claude Haiku 4.5 “more than five, and often more than ten”; in high-rethink games “only a small fraction of rethinks occur early in the game (typically around ≈ 10–20%)”; “One frequently observed failure mode occurs when a model is in check but proposes a move that fails to resolve the threat to their king”
- Toshniwal et al., “Chess as a Testbed for Language Model State Tracking” (AAAI 2022, arXiv:2102.13249, checked 2026-10-24): models trained on move sequences alone can learn to track pieces with enough data, and approximating attention over the full game history causes “a significant performance drop”
- The back-rank position in the picture and the role (White Kg1 Qd2 pawns f2 g2 h2, Black Kg8 Re1 pawns f7 g7 h7, White to move in check; only legal move Qxe1, checked with python-chess 2026-10-24), the opening line, the role wording and the mission prompt are illustrative, not from a measured game
- aifree.dev catalog, chatgpt-free (verified 2026-10-12, read-only API check 2026-10-24): ChatGPT Free Plan; Lichess analysis board (https://lichess.org/analysis) is free without an account
Published Oct 24, 2026 · updated Oct 24, 2026