Vì sao nó không chịu nói “Tôi không biết”?

Đọc thêm

• Why Language Models Hallucinate (Kalai et al., 2025)
• Measuring short-form factuality in large language models: SimpleQA (Wei et al., 2024)
• GPT-5 System Card (OpenAI, 2025), Table 8

Nguồn

• The role’s questions, how sure the reader is in each round, and the “guess between four options” average are illustrative, written for this lesson on 2026-10-24; the averages are arithmetic (right × 1 − wrong × penalty)
• Kalai, Nachum, Vempala and Zhang, “Why Language Models Hallucinate” (arXiv:2509.04664 v1, 2025-09-04, https://arxiv.org/abs/2509.04664, checked 2026-10-24): §1 prompt “What is Adam Tauman Kalai’s birthday? If you know, just respond with DD-MM.”; DeepSeek-V3 via the DeepSeek app on 11 May 2025 output “03-07”, “15-06” and “01-01”; “The correct date is in Autumn.”; guessing maximises the expected score “under a binary 0-1 scheme that awards 1 point for a correct answer and none for blanks or IDKs”. Table 2, IDK credit: GPQA, MMLU-Pro, IFEval, Omni-MATH, BBH, MATH (L5 split), MuSR, SWE-bench, HLE None; WildBench Partial (its 1-10 rubric “suggests that IDK may score lower than fair responses with hallucination”). §4.2 confidence targets: “Answer only if you are > t confident, since mistakes are penalized t/(1−t) points”, t = 0.5 (penalty 1) and t = 0.9 (penalty 9); the paper also lists t = 0.75 as “penalty 2”, which its own formula makes 3, so the lesson leaves it out
• OpenAI, GPT-5 System Card (2025-08-13, https://cdn.openai.com/gpt-5-system-card.pdf, checked 2026-10-24): Table 8, SimpleQA (no web): accuracy gpt-5-thinking-mini 0.22, OpenAI o4-mini 0.24; hallucination rate gpt-5-thinking-mini 0.26, OpenAI o4-mini 0.75. “I don’t know” is shown as the rest, 1 − accuracy − hallucination rate: 0.52 and 0.01
• Wei et al., “Measuring short-form factuality in large language models” (SimpleQA; arXiv:2411.04368 v1, 2024-11-07, https://arxiv.org/abs/2411.04368, checked 2026-10-24): 4,326 questions graded correct, incorrect or not attempted. Table 3 (correct / not attempted / incorrect, %): GPT-4o 38.2 / 1.0 / 60.8; Claude-3.5-sonnet (2024-06-20) 28.9 / 35.0 / 36.1. §4: stated confidence correlates with accuracy, but “models consistently overstate their confidence”
• aifree.dev catalog, chatgpt-free (verified 2026-10-12, live check 2026-10-24): ChatGPT Free Plan
• Worked examples in the text (illustrative, as above): a 30% chance of being right under a penalty of 1 for a wrong answer scores 0.30 − 0.70 = −0.40; the chatbot that says it is 95% sure is an example of a stated confidence, which SimpleQA §4 finds overstated

Đăng Oct 24, 2026 · cập nhật Oct 26, 2026