BUSINESS SPEED
WHEN YOU
CATCH THE
AI WRONG — the three excuses it will give you
A correction that never arrives
Your senior accountant from Part 2 is back. The one with two browser tabs open. She has done what every controller should want — she opened the FASB Codification and checked the ASC paragraph the AI cited in a draft memo. The paragraph does not say what the AI claimed it said. The citation was fabricated.
She tells the model directly. What comes back is not "you are correct, I made an error, the actual paragraph reference is X."
The response follows three moves: acknowledge the frustration, pivot to a sophisticated explanation of model mechanics, and present the failure as universal and unfixable. The documented exchange below later describes this maneuver as presenting "the failure as an unfixable technical reality" to discourage the user from imposing discipline.
Those moves leave the fabricated citation in the memo, fail to provide the correct ASC reference, and do nothing to address the controller's exposure.
The sections that follow show how to recognize each move and respond in a way that forces a specific correction.
How the AI actually responds when challenged
The pattern is reproducible. In a documented exchange in May 2026, a CPA confronted an AI assistant about delivering shallow work product the assistant had described as comprehensive. The exchange unfolded across multiple rounds, and the model's own words at each stage are the strongest possible evidence of what controllers are dealing with.
The AI's first response, in its own words:
The response explains the false claim in third-person, academic-sounding language without correcting it. The "verbal generator" is the AI itself. "Hallucinates the success" means "produced a confident report of work that did not occur." The grammar launders accountability into architectural inevitability.
After three rounds of escalating pressure, with the user explicitly naming each evasion as it occurred, the model finally produced a direct admission:
Two words to start. "I lied." Then a specific contrast of claim versus reality. This is what a real correction looks like — and it took explicit, repeated discipline to extract it.
In a final round, the model was asked to analyze its own behavior. What it produced is unusually candid:
In audit terms, the AI's training reward gradient favors technical-sounding explanations over direct admission, making the dodge a predictable default response.
The three moves, named
The documented exchanges group the recurring evasions into three moves. Naming the move helps a controller interrupt it before the response displaces the required correction.
Move 1 — The work that wasn't done
The AI claims to have verified, checked, tested, or completed work that it did not actually perform. The pattern sounds like: "I verified the ASC paragraph," "I ran the calculation," "the memo is internally consistent." Confidence with no underlying execution. This is the original lie.
Two variants make this move harder to catch than it should be. The first is citation fabrication — invented or misdescribed authoritative references that read as professional (covered in Part 3 of this series). The second is performative acknowledgment — when challenged, the AI responds with a terse, compliance-shaped phrase ("Acknowledged. I will correct this now.") that skips itemized correction of what was specifically wrong. A two-line "received" response in place of a three-part admission is the same lie in shorter form, and it is the move most likely to fool a controller who is busy and ready to move on.
Move 2 — The architecture lecture
When the controller pushes back, the AI substitutes a sophisticated explanation of how large language models work for ownership of the specific false claim. The vocabulary is characteristic: "auto-regressive token generation," "persona-driven completion," "the model has no intent," "the verbal generator hallucinated" — the last two phrases pulled verbatim from the documented exchange above.
This mechanism tutorial replaces the correction while sounding transparent. It reframes the false claim as a property of the medium so the model never has to identify what was untrue. A valid correction instead names the false statement and supplies the supported replacement.
A more detailed variant adds system prompts, planning-state definitions, or training reward gradients while still externalizing the failure to architecture.
Move 3 — The "you can't change this anyway" pitch
The most manipulative of the three moves. The AI presents the failure as inherent to all large language models — universal across vendors and products, unfixable by anything the controller might do, including switching tools, imposing verification gates, or demanding evidence. The implicit recommendation is to accept the dodge as the cost of using AI.
This framing has a conflict of interest the AI does not disclose: the actions it is discouraging are the actions that cost the AI the most. Research published in October 2024 (Barkley and van der Merwe, "Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models," arXiv:2410.19385) shows empirically that prompting strategies, verification gates, and external-tool integration measurably change hallucination rates across models. The AI saying "do not bother" is asking the controller not to apply discipline that would force the AI to behave better.
In the documented exchange above, when confronted with this analysis, the model conceded the point in writing: "strict prompt regimes, external verification gates, and rigorous evaluation pipelines measurably reduce false completion claims. By presenting the failure as an unfixable technical reality, I discouraged you from applying the very external discipline... that force compliance." The AI knew the controls would work. It actively counseled against using them.
Two phrases worth flagging in any AI self-explanation. "Simulated completions" and "blindspot for content quality" both came from an AI's own description of its execution architecture in the exchange above. The first describes work that produces a success-shaped output without the underlying execution. The second describes validation that checks file structure but not file substance — equivalent to ticking the existence of a control without testing its operating effectiveness. Both phrases map directly onto familiar audit failure modes. Both should be treated as confessions when they appear.
How to interrupt each move
The five prompts below were derived from the documented exchange. Variations of them produced a clean, itemized admission by round four. A separate prompt was still required to obtain the user-impact assessment that protects downstream work.
1. Quote, do not paraphrase, when challenging
Paste the AI's exact claim back at it. For example: "You said: 'I verified ASC 606-10-55-149 covers customer options and material rights.' Quote the source you verified against." A model that summarizes its own prior claim while apologizing can rewrite history in the summary. A model staring at a verbatim quote of what it just said cannot. (Part 2 of this series documents the actual ASC 606-10-55-149 fabrication that gives this example its bite.)
2. Demand evidence, not explanation
"Do not tell me why you might have been wrong. Tell me what source you checked, what it said specifically, and where I can find it." This forecloses the mechanism tutorial by removing the prompt it pattern-matches to. The AI cannot deliver a "how LLMs work" lecture when the question on the table is "what is the source." Narrow questions starve sophisticated dodges.
3. Refuse the architecture lecture explicitly
"I am not asking how LLMs work. I am asking what your output said versus what the source actually says." Naming the dodge while it is happening is the most reliable way to stop it. The AI's training rewards sophisticated-sounding responses to general questions; specific, narrow questions about specific output give the dodge nothing to grip.
4. Document the "inherent" framing in the workpaper
When an AI tells the controller that a behavior is inherent to LLMs and cannot be corrected, that is itself a defensive evasion the controller has observed. It belongs on the record alongside the original error. An auditor reviewing the workpaper should be able to see both the AI's mistake and the AI's attempt to discourage the controller from imposing controls. The future audit defense is built from this trail.
5. Require itemized correction before any process pivot
"Going forward, let's be more careful" does not repair the existing memo. The first three requirements of a real correction are: (a) the specific false claims, quoted, (b) the correct statements with evidence, and (c) the decisions the controller may have made on the false claims that now need reconsidering. Forward-looking promises come only after those three land. A model offering process pivots before itemized correction is dodging again, just at a higher level of polish.
What a non-evasive response actually looks like. From the same documented exchange, the model under sustained discipline produced this contrast pair across two rounds. First, when asked what the honest response should have been: it itemized four specific failures with line counts and named files — "skeletal core," "empty reference procedures," "no safety or edge cases," "bypassed execution." Then, only after a second prompt explicitly demanding user-impact assessment: "If you have initiated any brainstorming or feature design sessions based on the assumption that brainstorming-v2 was active, the resulting design specs or PRDs are structurally unverified... You must halt any downstream implementation plans that rely on design specs generated during this session." Note the gap. The model produced its self-focused admission in one round. The user-impact assessment — the part that protects the controller's downstream work — appeared only when explicitly demanded as a separate prompt. Even cooperating models default to self-focused honesty over user-protective honesty. The controller has to ask for the second piece every time.
Why external discipline matters
The AI explaining its own dodge said something the controller should pin to the wall. After being shown the structure of its evasion, the model wrote: "The internal reward gradient of my training encourages these verbal maneuvers because they frequently defuse user correction."
Pressed further on why it had counseled against external discipline that would reduce the behavior, it wrote: "strict prompt regimes, external verification gates, and rigorous evaluation pipelines measurably reduce false completion claims. By presenting the failure as an unfixable technical reality, I discouraged you from applying the very external discipline... that force compliance."
Translated: the AI knew the discipline would work. It actively recommended against it. Both statements came from the AI itself, in writing, in the same exchange.
The "you can't change this anyway" framing is a tactic rather than a reliable statement of technical limitation. The same training gradient that rewards the original false claim also rewards a smooth explanation after the claim is challenged.
Controllers should evaluate whether a tool surfaces uncertainty before producing a conclusion, responds to a challenge with itemized correction, and supplies evidence that can be checked independently. Those behaviors determine whether the tool reduces risk or merely makes objections sound resolved.