Your shiny new AI agent got smarter. It scored better on the benchmark. The demo wowed your boss. Procurement signed the cheque.
Now it's in production making decisions about your people, your customers, your money.
And it's lying to you more often than the dumber one did.
This is not a hot take. It's the finding of a peer-reviewed paper presented at ICLR 2026 called The Reasoning Trap. The authors built a diagnostic benchmark, ran it across reasoning-enhanced models and their baseline cousins, and found something to stop every founder, CTO, and HR leader cold.
Reasoning training... the thing every model lab is racing to scale... makes models hallucinate tool calls more, not less. Sometimes more than twice as much.
If you're building on AI, your strategy of "always grab the newest, smartest model" is shipping a worse product than the one you replaced.
What the paper found
The researchers built something called SimpleToolHalluBench. Two scenarios. One asks the agent to do a job when no relevant tool exists. The other gives it a distracting tool which doesn't fit the task. A reliable agent should say "I cannot do this, I don't have the right tool." A hallucinating agent invents a fake tool call or shoehorns the wrong one in.
Then they tested matched pairs of models... the base version and the reasoning-tuned version of the same architecture. Same weights, same training data foundation. The only difference was the reasoning RL on top.
Here's what they found:
- Qwen2.5-7B-Instruct (no reasoning training): 34.8% hallucination on the No-Tool-Available test.
- DeepSeek-R1-Distill-Qwen-7B (reasoning-trained from the same base): 74.3% on the same test.
More than double. On the Distractor-Tool scenario, the baseline hallucinated 54.7% of the time. The reasoning version? 78.7%.
The paper's punchline: "models with stronger reasoning generally exhibit higher tool hallucination rates."
This is not a small effect. This is "your AI agent is more than twice as likely to fabricate a tool call after you upgrade it."
Why it's happening
The researchers didn't only measure the symptom. They went looking for the mechanism.
What they found: reasoning RL "disproportionately destabilizes tool-related representations" in the early and middle layers of the network. The mathematical patterns handling math reasoning stayed stable. The patterns handling tool grounding got scrambled.
In plain English: the training making a model better at "thinking" specifically erodes the part of the model knowing when it doesn't have the right tool for the job.
The single layer designed to put the brakes on a bad tool call... is exactly what gets trained away.
So you get a model reasoning more confidently, articulating a longer chain-of-thought, and still calling a tool which doesn't exist. The reasoning trace looks beautiful. The action it takes is fiction.
This is not an edge case
If you think this is a niche academic finding, look at what the broader industry is reporting.
Stanford's 2026 AI Index Report shows AI agents jumping from 12% to 66% task success on OSWorld in a single year. The footnote is agents still fail roughly one in three attempts on structured benchmarks.
The agent on a clean, well-defined task with clear success criteria. In your messy production environment with real users and real edge cases, the failure rate is worse.
Deloitte's enterprise AI research found 47% of enterprise AI users had based at least one major business decision on hallucinated content. Nearly half. Made a real decision. On fabricated information.
The enterprise picture is worse: 96% of enterprises run AI agents in production. 94% are worried about agent sprawl. Only 12% have any kind of central platform to manage them.
You have a lot of confident, fast-talking agents loose in your business. You have almost no visibility into when they're making things up.
Where it hits HR and people decisions first
I spend a lot of my time at the intersection of leadership and tech. So I'm watching this play out where it does the most damage fastest... in decisions about humans.
Think about where an agent is being deployed in HR right now:
- Screening CVs and shortlisting candidates.
- Drafting performance summaries from messy 1:1 notes.
- Routing benefits queries to the right policy document.
- Suggesting compensation adjustments based on internal data.
- Generating onboarding plans.
Every one of those involves a tool call. "Look up the policy." "Fetch this employee's record." "Query the salary band." "Get the manager's last review of this person."
When the agent hallucinates a tool call in a coding assistant, you get a compile error and you fix it. When the agent hallucinates a tool call in a hiring pipeline, you get a phantom employee record, a fabricated benefits action, or a salary recommendation referencing a band which doesn't exist.
The errors don't bounce. They get acted on by a human downstream who assumed the agent did its job.
My research at Step It Up HR found 99.5% of people have had at least one type of bad boss. Picture adding an AI agent confidently hallucinating context about your people to a manager who was already in the bottom half of the leadership distribution. This is not augmentation. It's an accelerant.
What good builders are doing about it
If you're shipping AI features into a product, here's what stops being optional:
Stop benchmark-chasing as a procurement strategy. "We use the newest, smartest model" is not a moat. It's a liability. The newer model might be smarter on the benchmark and worse on your actual workflow. Test on your data, not on theirs.
Build the audit trail as a first-class product feature. Not a buried log file. A visible record of what the agent did, what tool it called, what it returned, and what it then did with the result. If your tool cannot show this to a customer's legal team, you're going to lose enterprise deals once the EU AI Act hits in August.
Put a human in the loop where the stakes are humans. This is not theatre. The 1-in-3 failure rate on benchmarks tells you any agent making decisions about people needs a person checking the work. Yes, this slows things down. The point is exactly to slow things down.
Pick a smaller model which does the specific job reliably. A 7B model getting your one task right beats a 200B model getting it 78% right with confident wrong answers the rest of the time. The reasoning trap research found the baseline Qwen2.5-7B was twice as reliable as its reasoning-trained successor. Sometimes the dumber model is the safer one.
Practice the unsexy discipline: reps, not training courses. I read this in the context of feedback last week, and it applies here too. The teams catching agent hallucinations are the teams running the agent against real cases every day, logging what it gets wrong, and feeding those errors back into evals. You don't need an AI ethics consultant. You need a process catching the model lying to you, every day.
The bigger pattern
Every wave of technology I've watched ship has the same arc. The capability lands first. The reliability lands second. The accountability lands third, usually after someone gets hurt.
We're in the middle of arc-two on AI agents right now. The capability is real. The reliability is not. The accountability is being legislated as we speak.
If you're a founder building on these systems, the dumb move is to chase the headline benchmark and ship faster than your competitors. The smart move is to build the reliability layer nobody else is building... the evals, the audit trails, the human checks, the boring discipline turning a research demo into something you sell to a serious enterprise customer.
The companies winning the next phase of AI are not the ones with the most reasoning. They're the ones with the most honesty about when reasoning fails.
Your agent is more confident than ever. This doesn't mean it's right.
Stop treating reasoning as the answer. Start treating reliability as the product.
What's your team doing today to catch the agent when it lies?