Why does it give me links that don't exist?
Fake links: without search, a chatbot writes a URL or a reference like any other text, one likely piece at a time (a familiar kind of site, a plausible path, a year, words from the question), so it looks real but nothing checks that the page exists; with search on, links come from real results but still need a click; ask for the quoted sentence and open the link or DOI yourself · Beginner · 5 min · sampling, rag · after Web search and grounding: snippets in, a link per sentence out, Knowledge cutoff: training data stops at a date
Fake links happen because a chatbot writes a URL like any other text: one likely piece after another. Each piece fits, but nothing checks that the page exists. Search helps; opening the link yourself settles it.
1 Guess first
2 Look
A link, as you get it and as the model writes it
1 / 5The chatbot’s answer
Here is a study: “Sleep boosts memory consolidation”, Journal of Sleep Studies, 2021.
https://journal.example.org/articles/2021/sleep-boosts-memory-consolidation
You open it
Page not found
illustrative: example.org is reserved and leads nowhere
You asked for a source and got a neat link with a journal, a year and a title. You open it: nothing there (illustrative).
A link is written like any other sentence, one likely piece after another; every piece fits, and no step checks that the page is there.
A chatbot writes a link the way it writes a word: it predicts the next token, a small chunk of text, from what usually comes next. A journal name makes a journal site likely, then a year, then words from your question. Whether the whole address leads anywhere is never part of the guess.
4 You are the model
You are the chatbot, with no search. Someone asks for a source that sleep helps memory. Write the link one piece at a time.
Asked: “Give me a source that sleep helps memory.” Your link so far
https://
No search. Piece 1 of 4: Which site? (a journal is likely)
illustrative: example domains are reserved and lead nowhere
You are the chatbot. Write the link one piece at a time.5 Check
Turning on web search, the chatbot looking pages up first, changes where links come from: it can copy them from real results. It still slips. In a 2026 study, 4.8% of URLs cited by chatbots with search had never existed; for deep research agents, which cite far more, 10.7%.
Three habits help. Turn search on when you need sources. Ask for the quoted sentence, the exact words from the page, so you can find them. Then open each link, or paste the DOI into doi.org, before you trust or share it. In the same study, letting GPT-5.1 check its own links cut dead ones from 16.0% to 0.6%.
8 Say it back
9 Mission · 5 min
In ChatGPT Free, start a new chat and ask “Without searching the web, give me 3 links to studies showing that sleep improves memory. Full URLs only.” Open each link: how many work? Then ask again with Search turned on, adding “Quote the sentence from each page that supports it.” Open them again: how many work now, and is each quote on its page?
Three quick ones
Drawn at random from this lesson's pool; different next time.
Done?
Get an email when new lessons come out
New lessons arrive in the weekly digest, only in a week that has one. You confirm by email first; unsubscribe in one click.
Go deeper
- Fabrication and errors in the bibliographic citations generated by ChatGPT (Walters and Wilder, 2023)
- Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents (Rao et al., 2026)
- LLM hallucinations in the wild: Large-scale evidence from non-existent citations (Zhao et al., 2026)
Sources
- Walters and Wilder, “Fabrication and errors in the bibliographic citations generated by ChatGPT” (Scientific Reports 13:14045, published 2023-09-07, https://doi.org/10.1038/s41598-023-41032-5, full text on PMC10484980, checked 2026-10-24): ChatGPT-3.5 and ChatGPT-4 each wrote 42 short papers, texts “generated in the first week of April 2023”. “55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated” (Table 3: 55% of 222, 18% of 414 cited works); “43% of the real GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors”
- Rao, Wong and Callison-Burch, “Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents” (arXiv:2604.03173 v1, 2026-04-03, https://arxiv.org/abs/2604.03173, checked 2026-10-24): a URL is “hallucinated” when it does not resolve and has no Wayback Machine record (“likely never existed”); “3–13% of citation URLs are hallucinated”. §4.2: “the hallucination rate is 10.7% [10.2, 11.2] versus 4.8% [4.3, 5.2] for the eight search-augmented models” (two deep research agents pooled vs eight search-augmented models, DRBench). §5.1, with the urlhealth link checker as a tool: “GPT-5.1 from 16.0% to 0.6%” non-resolving
- Tokens in the picture: OpenAI o200k_base (the GPT-4o and GPT-5 tokenizer in tiktoken’s model map), computed with tiktoken 0.14.0 on 2026-10-24: the 75-character made-up link is 18 tokens
- The chatbot answer, the made-up link, the role’s link pieces and search results are illustrative, not recorded model answers; they use reserved example domains (RFC 2606), so they never point at a real site. The DOI in the last state is the real Walters and Wilder paper
- aifree.dev catalog, chatgpt-free (verified 2026-10-12, live check 2026-10-24): ChatGPT Free Plan
Published Oct 24, 2026 · updated Oct 24, 2026