The strongest argument against putting an AI agent into a production workflow is that language models make things up. The objection is correct, and it is the right thing to worry about. A model that invents a vendor name on a payment file, or a balance on a reconciliation, is worse than no automation at all, because a wrong answer delivered with confidence is harder to catch than a blank field. What makes the objection misleading as a blanket disqualifier is that the hallucination rate is not a fixed property of the model. It is a property of the model, the specific task, and whether the output is grounded in retrieved source text and checked against that text before anyone acts on it. The same model can sit at opposite ends of a reliability range depending on how the work is set up, and the size of that swing is the part most operators have not seen measured.
Two numbers from the same kind of model
Two bodies of research put the range in concrete terms. Vectara maintains a Hallucination Leaderboard that scores models on a grounded summarization task: the model is given a source document and asked to summarize it, and a detector checks whether the summary is supported by the source. On that task, the leaderboard's May 2026 update puts the best model's rate at 1.8 percent, with the next best models a little above three percent. Tamber and colleagues at Vectara describe the leaderboard's methodology in a paper presented at the EMNLP 2025 Industry Track, which treats hallucination as a measurable rate on a defined task rather than a vague tendency.
The Stanford RegLab figures come from a different setup with the same class of model. In a 2024 study in the Journal of Legal Analysis, Dahl, Magesh, Suzgun, and Ho tested general-purpose models on legal queries and documented hallucination rates of fifty-eight percent to eighty-eight percent when the model answered from its own parametric memory. In that mode the model was asked a question and produced an answer from what it had absorbed during training, with no source document in front of it. The two numbers can be set side by side. A capable model summarizing a document it has been handed errs two or three times in a hundred. A capable model answering a question from memory errs more often than not. The class of model did not change between those two studies. The task did.
The mechanism that separates them
The difference between those numbers is retrieval grounding. In the ungrounded case, the model is a compression of its training data, and when asked for a specific fact it reconstructs a plausible answer from statistical association. Sometimes that reconstruction is correct and sometimes it is a fluent invention, and the model has no way to tell you which. In the grounded case, the relevant source text is retrieved and placed in front of the model, and the task narrows from recall to reading. The model is no longer asked what it knows; it is asked what the document in front of it says. That is an easier task to get right, and an easier task to check.
Checking is the second piece. Once the output is supposed to be supported by a specific passage, you can verify it against that passage, which is what Vectara's detector does to produce its rate. In a production workflow the same idea becomes an output-verification step: the agent's proposed answer is compared against the retrieved source before it is committed, and a claim with no supporting passage is rejected rather than passed through. The third piece is what happens when verification fails or confidence is low. Rather than guessing, the workflow refuses or escalates, handing the item to a person. We treat that exception path as the design that makes the rest of it safe to run, the same posture we take in how production reliability gets engineered.
Why bounding the task matters more than picking the model
The reason this works in operations is that the tasks worth automating are mostly grounded by nature. A three-way match reads a purchase order, a receipt, and an invoice that all exist as documents. A collections agent works from an aging report and an open invoice, both retrievable. Month-end reconciliation ties one ledger to another, and both ledgers are there to be read. These are reading-and-checking tasks, not recall-from-memory tasks, which puts them on the side of the range Vectara measures rather than the side Stanford measured. The ungrounded mode that produces the high rates is the mode a well-built workflow does not use. The model is never asked to remember a customer's balance; it is shown the balance and asked to act on it.
This connects to a point we have made about the limits of automation. The residual error in a grounded, verified workflow concentrates in the genuinely ambiguous cases, the long tail of document variability where the source itself is unclear, and those are the cases the exception path exists to route to a person. It also bears on the harder question of when not to automate at all. A task that cannot be grounded in retrievable source text, where the answer genuinely has to come from the model's memory, is a task where the high rates apply and the honest answer is to keep a person in the seat.
The honest counterpoint
A reasonable counter is that newer is not always safer. Reasoning-tuned models, the ones trained to work through problems in multiple steps, do not uniformly hallucinate less than the models they replaced, and the Vectara work itself shows that some of them score worse on the grounded task than their predecessors did, with less consistency across runs. So the objection cannot be answered by pointing at a frontier model and assuming the rate falls as the version number rises. There is real substance here. The rate is not monotonic in model capability, and a vendor who tells you a problem is solved because they switched to the latest model is not measuring what we just described.
The answer is that this is an argument for measurement rather than against deployment. If the rate depends on the model, the task, and the grounding, then the rate is something you test on your own workflow before you trust it, the way you would test any control, which is the same discipline behind the 80-to-99% problem of getting an agent from a working demo to production reliability. Hallucination in a bounded, grounded, verified workflow with a person on the exception path is a measured quantity. It has a rate you can put a number on, a verification step that catches most of what slips through, and an escalation path for the rest. The objection is right that models make things up. It is wrong only when it treats that as a fixed reason to do nothing, because the setup determines the rate, and the setup is something we control.