Vì sao nó không nói đúng một ngày rơi vào thứ mấy?

Đọc thêm

• Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning (Fatemi và cộng sự, 2024)
• Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning (Bhatia, Peyrard và Zhao, 2025)
• Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMs (Saxena, Gema và Minervini, 2025)

Nguồn

• Fatemi, Kazemi, Tsitsulin, Malkan, Yim, Palowitch, Seo, Halcrow and Perozzi, “Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning” (arXiv:2406.09170 v1, 2024-06-13, https://arxiv.org/abs/2406.09170, checked 2026-10-24): ToT-Arithmetic, 1,850 questions (350 AddSubtract, 350 Compare, 200 Duration, 250 Schedule, 100 Timezone, 250 Trick, 350 MultiOp). Table 8, accuracy Claude-3-Sonnet / GPT-4 / Gemini 1.5 Pro: AddSubtract 58.57/76.28/71.14, Compare 39.14/63.14/55.43, Duration 15.00/16.00/13.50, Schedule 29.60/43.60/40.00, Timezone 74.00/88.00/90.00, Trick 40.40/45.60/41.20, MultiOp 26.57/46.86/62.57. §4.4: “the most common error among incorrect answers was a deviation of precisely one day from the ground truth label”; “Common failure: leap year calculation”. Table 2 Duration example quoted in the predict step
• Saxena, Gema and Minervini, “Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMs” (arXiv:2502.05092 v2, 2025-03-18, https://arxiv.org/abs/2502.05092, checked 2026-10-24): CalendarQA, a full-year calendar image and a weekday question, 10 years × 6 questions. Table 1, Calendar Acc: Llama 3.2-Vision 11.67, Qwen2-VL-7B 18.33, MiniCPM-V-2.6 20.0, Gemini-2.0 31.67, GPT-4o 43.33, Claude-3-5-sonnet 46.67, GPT-o1 80.0
• Miao, Fu and Wei, “SPAN: Benchmarking and Improving Cross-Calendar Temporal Reasoning of Large Language Models” (arXiv:2511.09993 v2, 2026-01-09, https://arxiv.org/abs/2511.09993, checked 2026-10-24): calendar questions across six calendars, evaluation dates 1960–2060. Figure 6 and §Results, average accuracy: Time Agent (GPT-4o writing code that a code interpreter runs, with a calendar-conversion tool) 95.31%, OpenAI-o1 59.29%, GPT-4o w/ RAG (Bing search) 43.69%, GPT-4o 43.01%. Abstract: “these LLMs achieve an average accuracy of only 34.5%”; “Future-Date Degradation”
• Bhatia, Peyrard and Zhao, “Date Fragments: A Hidden Bottleneck of Tokenisation for Temporal Reasoning” (arXiv:2505.16088 v3, 2025-09-24, https://arxiv.org/abs/2505.16088, checked 2026-10-24): BPE tokenisers split dates into fragments; “excessive fragmentation correlates with accuracy drops of up to 10 points on uncommon dates like historical and futuristic dates”; Table 3, date arithmetic accuracy GPT-4o-mini 56.67
• Cheng, Marone, Weller, Lawrie, Khashabi and Van Durme, “Dated Data: Tracing Knowledge Cutoffs in Large Language Models” (arXiv:2403.12958 v2, 2024-09-17, https://arxiv.org/abs/2403.12958, checked 2026-10-24): “effective cutoffs often drastically differ from reported cutoffs”; one reason: “temporal misalignments of CommonCrawl data due to non-trivial amounts of old data in new dumps”
• Tokens measured 2026-10-24 with tiktoken 0.14.0, encoding o200k_base (GPT-4o): “2026-10-24” 6 [1323, 21, 12, 702, 12, 1494] = “202” “6” “-” “10” “-” “24”; “24/10/2026” 6 = “24” “/” “10” “/” “202” “6”; “24 October 2026” 5 = “24” “ October” “ ” “202” “6”; “20261024” 3 = “202” “610” “24”
• Every weekday in the lesson is computed by code (JavaScript Date, Gregorian calendar, UTC) and checked by its unit test: Thursday 1 January 2026, Sunday 1 March 2026, Saturday 24 October 2026, Friday 25 December 2026, Wednesday 28 February 2024, Friday 1 March 2024, Friday 1 January 2100; 24 October 2023–2030: Tuesday, Thursday, Friday, Saturday, Sunday, Tuesday, Wednesday, Thursday
• The “today” chat, the role’s rounds, the mission prompt and the habit wording are illustrative, written for this lesson on 2026-10-24; the date sums in them are checked by code
• aifree.dev catalog, chatgpt-free (verified 2026-10-12, live check 2026-10-24): ChatGPT Free Plan
• Calendar arithmetic in the text (Gregorian calendar, checked by the lesson’s unit test): 1 January to 24 October 2026 is 296 days = 7 × 42 + 2; a year of 365 days is 52 weeks and 1 day, a leap year of 366 days 52 weeks and 2 days; October has 31 days; 2000 divides by 400, so it kept its 29 February; a random weekday is right one time in seven, 14.3%. “When William was 400 days old” is the Test of Time Duration example (Table 2) quoted in the predict step

Đăng Oct 24, 2026 · cập nhật Oct 26, 2026