What does it actually mean when an AI 'hallucinates'?

It means the AI made something up and presented it as a fact. Think of it like a student who didn't study for a test. Instead of admitting they don't know the answer, they write down something that sounds plausible but is completely wrong. AI models don't 'know' things the way we do. They predict the next most likely word in a sequence. Sometimes, the most likely sequence sounds right but has zero connection to reality. I've seen this trip up a lot of new users. You ask for a biography of a minor historical figure, and the AI confidently invents a birthplace, a degree they never earned, and a quote they never said. It's not lying. Lying requires intent. The model simply has no internal fact-checker. A famous example from early 2023 involved a lawyer who used ChatGPT to prepare a court filing. The AI cited multiple previous court cases to support his argument. The problem? None of those cases existed. The AI had generated realistic-sounding case names and rulings out of thin air. The lawyer was sanctioned by the court. This is why you can't blindly trust AI output for anything important. Always verify facts from a primary source. A good mental model is to treat the AI like a very smart, very fast intern who sometimes gets a little too creative. You love their speed, but you double-check their work before it goes out the door.