AI tools make things up because they are trained to produce the most plausible next chunk of text, not the most accurate one — and nothing in that process checks whether the output is actually true.
Fluency and truth are separate things, and a language model is optimized only for the first. When you ask a chatbot a question, it does not look up an answer in a database. It predicts, word by word, what a helpful-sounding response would look like, based on patterns absorbed from enormous amounts of text. A fabricated citation, a made-up statistic, or a feature that does not exist can come out in exactly the same confident tone as a correct answer, because the model has no internal flag that separates the two.
This is what people mean by an AI hallucination, and it is a structural feature of how these systems work, not a bug that a future update will simply erase.
The mechanism is worth understanding because it explains when you should be most suspicious. During training, the model sees billions of examples of text and adjusts itself to get better at one job: guessing what comes next. It is never rewarded for saying "I don't know."
It is rewarded for producing text that fits the pattern. So when a question calls for a specific fact — a date, a study, a page number, a price — the model generates whatever string of characters best fits the shape of the answer, even if no such fact exists. Confidence is not calibrated to correctness.
A model can be equally sure about the capital of France and about a research paper it invented, because in both cases the output simply "looks right" to the prediction process. That is why hallucinations cluster around specifics: names, numbers, quotes, citations, and niche details where the training signal was thin or the correct answer is rare.
Vague, well-trodden topics tend to be safer; precise, obscure ones are where fabrication thrives.
Here is a concrete case. Ask a chatbot to summarize recent research on, say, AI in medical imaging and to include citations. It may produce a tidy paragraph with three references — author names, a journal, a year, a volume number — that reads like a real bibliography.
Often one or more of those references do not exist. The author name is plausible, the journal is real, the year is reasonable, and the combination is invented. Nothing in the model's design produced that citation by looking it up; it produced it by predicting what a citation in that context would look like.
The same thing happens with product features. Ask about a tool's plan limits and the model may confidently describe a tier or a usage cap that the vendor has never offered. According to our AI tool database, which tracks 360 AI tools with a pricing and capability snapshot recorded at verification time, even simple facts like a plan price are the kind of detail that changes and must be checked at the source — the database lists Notion AI, for instance, as an $8 per user per month add-on, but a model answering from memory could easily state a different figure with total confidence.
The lesson is not that the model is lying. It is that the model was never doing the thing you assumed it was doing.
So how do you tell when an answer is wrong? The honest answer is that you often cannot tell from the text alone, which is why you should not try. Treat any specific claim — a number, a quote, a citation, a date, a feature — as unverified until you check it somewhere that actually stores facts.
A practical rule: the more precise and the more obscure the claim, the more it needs a source you can open. If a chatbot cites a paper, search for the paper. If it states a price, open the vendor's page.
If it describes a feature, look for it in the product's own documentation. Retrieval-augmented tools, which pull passages from a real document set and cite them, reduce fabrication because the model is now summarizing something in front of it rather than recalling from memory — but they do not eliminate it.
The model can still misread the passage, blend two sources, or cite a real document while stating something the document does not say. The advice also does not apply evenly: creative tasks like brainstorming names or drafting an email have no ground truth to be wrong about, so fabrication matters far less there.
It matters most when the output will be acted on — a medical claim, a legal citation, a financial figure, a line of code. In those cases, the model is a drafting assistant, not an authority, and the check is yours to do. A useful habit is to ask the model to flag its own uncertainty and to name its sources, then verify the sources independently; that turns a confident paragraph into a list of claims you can actually test.
For a deeper look at why these tools fail in ways that feel unpredictable, see our explainer on why AI sometimes makes things up and how to tell when it's wrong.