Why does it reply in the wrong language?
Wrong-language replies (language confusion): a chatbot learned mostly from English text, and chat training was mostly English, so English words often stay likely next pieces even when you write in another language; an English system prompt, a pasted English document or technical terms raise them further, and the reply slides into English or mixes in English words; say the reply language at the start and the end, keep the system prompt in your language, and lower the temperature where you can · Beginner · 5 min · sampling, llm · after Tokenizers and languages: why the same sentence costs more, System prompts and instruction priority
A chatbot replies in the wrong language because English words often stay likely next pieces: it learned mostly from English. English instructions or pasted English text make them likelier. Say your language at the start and the end.
1 Guess first
2 Look
A Vietnamese question, and an answer that slides into English
1 / 6You ask
Chỉ số Gini được tính thế nào?
(How is the Gini index calculated?)
The chatbot answers
- Chỉ số Gini đo mức bất bình đẳng thu nhập.Vietnamese · “The Gini index measures income inequality.”
- Nó dựa trên một đường cong called the Lorenz curve.one English word slips in · “It is based on a curve called the Lorenz curve.”
- The index is the area between that curve and the line of equality.a whole line in English
illustrative: not a recorded model answer
You asked in Vietnamese. The answer starts in Vietnamese, slips in one English word, then finishes a whole line in English (illustrative).
At each step English words are still among the likely next pieces; one of them gets picked, and the reply slides into English.
A chatbot writes one token, a small chunk of text, at a time, picking from the likely next ones. It learned from text that is mostly English: Llama 2’s training text was 89.70% English and 0.08% Vietnamese. So even in a Vietnamese answer, an English word is often a likely next piece.
4 You are the model
You are the chatbot. Three times you are partway through a Vietnamese answer. Read what you were given, then pick the next words.
Round 1 of 3. What you were given
- You get: “Chỉ số Gini dựa trên gì?” (What is the Gini index based on?)
Your answer so far: “Nó dựa trên một đường cong ___” (It is based on a curve ___). Which words come next?
illustrative shares, not recorded model output
You are the chatbot. Pick the next words of your Vietnamese answer.5 Check
Chat training matters too. The base Llama 3 70B, trained only to continue text, stayed in Vietnamese 97.0% of the time; after chat training on mostly English examples, 10.0%. Newer models do far better, but none is perfect: in a 2025 test, Qwen3-235B slipped Latin letters into 5.07% of translations, GPT-5-Chat into 0.67%.
Three habits help. Name the reply language at the start and the end of a long prompt; buried in the middle, it worked worst (69.0% vs 86.7% at the start for one model). Write the system prompt, the instructions the chatbot gets first, in your language. Where you can set it, lower the temperature.
8 Say it back
9 Mission · 5 min
In ChatGPT Free, start a new chat. Paste a paragraph of English news, then below it ask in another language you know, such as Vietnamese or Spanish, for a three-sentence summary. What language is the reply in, and are English words mixed in? Then start a new chat and add “Reply only in <your language>” at the start and again at the end, written in that language. Does it stay in your language now?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Get an email when new lessons come out
New lessons arrive in the weekly digest, only in a week that has one. You confirm by email first; unsubscribe in one click.
Go deeper
Sources
- Marchisio et al., “Understanding and Mitigating Language Confusion in LLMs” (EMNLP 2024; arXiv:2406.20052 v3, 2025-04-03, https://arxiv.org/abs/2406.20052, checked 2026-10-24): Language Confusion Benchmark, 15 languages incl. Vietnamese, at most 100 tokens per reply. Line-level pass rate (LPR) = “percentage of model responses that pass our line-level language confusion detector without error. A response is “correct” if all lines match the user’s desired language.” Table 3, monolingual LPR, vi: Llama 3 70B-I 10.0, Llama 2 70B-I 17.0, Llama 3.1 70B-I 100.0, GPT-4o 99.0. Table 6, vi: Llama 2 70B base 98.0, Llama 3 70B base 97.0; §4.5: “instruction-tuned Llama models are much more confused, indicating English-centric instruction tuning”
- Marchisio et al. 2024, Table A11 (cross-lingual LPR by position of the language instruction), Command R: Start 86.7, Integrated 69.0, End 85.1; §4.3: “Integrated instructions cause more confusion: Command R has only 69% LPR on this type of prompt, versus 85% for isolated types”. Table 8 (monolingual word-level pass rate of Command R, ar/hi/ja/ko/ru/zh average): T=0.0 97.2, T=1.0 86.5
- Touvron et al., “Llama 2: Open Foundation and Fine-Tuned Chat Models” (arXiv:2307.09288, 2023, checked 2026-10-24), Table 10, language distribution in pretraining data: en 89.70%, vi 0.08% (“Most data is in English”); about 1,100 times more English than Vietnamese (calculated: 89.70 / 0.08 ≈ 1,121)
- Zhang, Huang, Yuan and Lin, “Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation” (arXiv:2510.17555 v1, 2025-10-20, https://arxiv.org/abs/2510.17555, checked 2026-10-24): §3.1, Qwen3-8B on FLORES-NO-LATIN: “the confusion token is the top-1 token 56.74% of the time” and “language consistent tokens appear within top-3 99.29% of the time”. Table 6, commercial chat models on FLORES-NO-LATIN (translations from English whose references contain no Latin characters, “so any Latin script in model output is considered erroneous”; Latin % = replies with erroneous Latin characters): Qwen3-235B-Instruct 5.07, GPT-5-Chat 0.67, Claude-Sonnet-4 0.35
- The chat, the reply, the next-word scores and the role’s shares are illustrative, not recorded model output. The picture’s shares at temperature 0.3 are calculated from the illustrative ones at 1.0 (softmax of log-share / temperature)
- aifree.dev catalog, chatgpt-free (verified 2026-10-12, live check 2026-10-24): ChatGPT Free Plan
Published Oct 24, 2026 · updated Oct 24, 2026