AI makes things up because it is not looking anything up — it is generating the next words that fit the pattern of what it has seen, so a fluent, confident sentence can be entirely invented.
The model is predicting plausible text, not retrieving verified facts, which means confidence and accuracy are two separate things. That is the whole explanation in one line, and everything useful about spotting it follows from there.
## Why confidence is not a signal of accuracy
When you ask a language model a question, it does not search a database and hand you a stored answer. It produces words one at a time, each chosen because it fits the pattern of the words before it. If your question looks like the kind of question that usually has a name, a date, or a citation attached, the model will produce a name, a date, or a citation — because that is what the shape of the sentence calls for.
The failure mode is easiest to see with specific, low-frequency details. A model asked about a well-known event will usually get the broad strokes right, because those patterns are dense. Ask about a niche court case, a small company's pricing page, or the exact wording of a local regulation, and the pattern is thin.
The model still produces a full, grammatically perfect answer. It just fills the gaps with whatever sounds most like the surrounding text. This is why hallucination clusters around names, numbers, quotes, and citations — the four things a reader is most likely to trust without checking.
## What it looks like in practice
A concrete example makes this click. Suppose you paste a meeting transcript into an AI tool and ask it to pull out action items. The transcript says: "We should probably revisit the onboarding flow at some point — someone needs to own that."
No name is attached. The tool returns: "Action item: Priya to review onboarding flow by Friday." Priya was in the meeting.
Friday was mentioned earlier about something else. The model did not lie on purpose and it did not check anything. "Tasks have owners" and "owners get deadlines" are extremely strong patterns in business writing, so the model completed the pattern. The transcript never said Priya, and it never said Friday.
This is the same mechanism behind a fabricated statistic in a research summary or a made-up product feature in a comparison table. The sentence needed a number, so a number appeared.
## How to tell when it's wrong
You cannot detect a hallucination by tone. Fluent and wrong sounds exactly like fluent and right. What you can do is sort the output by how checkable each claim is.
- Names, dates, prices, quotes, and citations — treat every one as unverified until you find it in a primary source. These are the highest-risk categories.
- Broad explanations and summaries of ideas — usually reliable enough to use as a starting point, because they lean on dense patterns.
- Anything about your own private documents — check against the source text directly. If the tool claims a detail that is not in the document you gave it, that detail was generated, not extracted.
A useful habit: ask the tool to quote the exact sentence from your source that supports each claim. If it cannot produce one, the claim is probably pattern completion. This does not make the tool useless — it makes it a drafting assistant rather than an authority. Tools like Notion AI and Slack AI are built around summarising and searching content you already have, which is a narrower and safer job than answering open questions about the world, but the same rule applies: the summary is a claim, not a receipt.
## Where this advice breaks down
Being skeptical of everything has a cost. If you verify every sentence, you have not saved any time, and you have added a step. The practical trade-off is to verify in proportion to the stakes. A brainstormed list of blog titles needs no checking. A number going into a client report needs a primary source. A medical or legal claim needs a human professional, full stop.
There is also a limit to how well you can catch errors in a domain you do not know. If you cannot tell a real citation from a fake one, no amount of careful reading will help — you need a second source, not sharper eyes. And note that some tools reduce hallucination by grounding answers in documents you supply, which helps for questions about those documents and does nothing for questions about the wider world.
For a deeper look at how these failures happen, see What is an AI hallucination and why do AI tools make things up? and Why does AI sometimes make things up and how can I tell when it's wrong?.