AI hallucination
An AI hallucination is a fluent, incorrect answer produced by a language model that presents invented facts or citations as if they were true, with no warning that the content is unreliable.
An AI hallucination is when a language model produces text that sounds correct but is not. The system doesn't look up a verified record the way a database query does. It predicts the next likely words based on patterns in its training. That process can assemble a plausible invoice number or a policy clause that was never in the source. The output is fluent, so the error is easy to miss.
In a business, this appears wherever a model writes or extracts information without checking your own records. An AI helpdesk can invent a password reset step that is not in your process. Document tools can fill a missing field with a plausible value. An agent preparing a report may add a figure that looks consistent with the rest of the page. Staff may then treat a confident answer as a source of truth.
The damage arrives when invented details enter a finance system or go out in a customer reply. Connecting the model to your own files reduces invention, because it has something to cite. That still isn't a complete defence. Human review belongs on any step that moves money or changes a record. Limits on what the model can write back into systems stop an unchecked error becoming an operational one.
When it matters
- →When AI answers are copied into finance records or customer emails.
- →When an agent can update live systems without a check.
- →When staff accept AI answers without checking the source.
- →When documents have missing fields the model may invent.
Related terms
AI hallucination: common questions
Why do AI models hallucinate?
Language models generate the next likely words from patterns in their training, not by fetching a verified record. When they lack a reliable source, they still produce a complete, confident answer. That answer can include invented facts or citations. The behaviour is a product of how the model writes, not a glitch that a restart will fix. Retrieval from your own files and human review both reduce how far those errors travel.
How can a business reduce AI hallucination?
Start by limiting what the model can do. Give it your own documents and records to draw from, rather than asking it to answer from general training. Set review points on any output that updates a system or goes to a customer. Keep the model from updating records until a person has checked the result. Prompt design helps, but it doesn't remove the risk on its own. Treat hallucination as an operational control problem, not a wording problem.
Can AI hallucination affect business decisions?
Yes. A wrong figure in a summary or an extracted contract date can be copied into a live process before anyone notices. The model won't flag the invention as uncertain unless you build that check in. For operations, the question is where the output goes next. If it stays on screen for a person to verify, the cost is time. If it updates a finance record or a customer file, the cost is a wrong action.
Ready to talk?
Book a free, no-obligation discovery call. We'll learn about your business and show you exactly how Wanzo can help — with a bespoke proposal within 48 hours.
No commitment. No sales pressure. Just honest advice.