AI Hallucination
A hallucination is a confidently delivered but invented answer: a quotation, link, paper or figure that does not exist. It is not a malfunction but a direct consequence of the design: the model predicts a plausible continuation rather than verifying a fact.
Why it happens
The model is trained to continue text plausibly. It has no separate mechanism distinguishing "I know this" from "this is how such things are usually written". Asked about a paper that does not exist, it produces what a reference to such a paper would look like.
It happens most where training data was thin: narrow topics, precise figures, names, dates, specific links. And almost never where the needed text sits directly in the request.
The dangerous part is that the tone does not change. An invented citation arrives with the same confidence as a correct one, so the text itself gives you no way to tell them apart.
How to reduce the risk
- Supply the source in the requestWith the text in front of it there is nothing to invent. This is the most reliable technique, and RAG is built on it.
- Ask for the quote, not the gistDemanding a verbatim fragment from the provided text sharply narrows the room for invention.
- Permit "I don't know"An explicit instruction to say "no data" instead of guessing changes the behaviour noticeably.
- Verify anything verifiableOpen the links, check the numbers against the primary source, find the quotes in the original. Especially when they look convincing.