I Accuse: How Irresponsibility, Pseudoscience, and Our Own Cognitive Bias Built the AI Delusion

In the tradition of Émile Zola’s J’Accuse — a formal indictment of a system that has failed the truth

“I have but one passion: to enlighten those who have been kept in the dark, in the name of humanity which has suffered so much and is entitled to happiness.” — Émile Zola, L’Aurore, January 13, 1898

On January 13, 1898, Émile Zola published an open letter that shook the French Republic. He named names. He stated charges. He accepted the legal consequences. What made J’Accuse immortal was not its eloquence but its refusal to be polite in the face of an organized lie.

I am not Zola. The injustice I describe will not send anyone to Devil’s Island. But it is an injustice nonetheless — one perpetrated not by a single corrupt institution but by an entire ecosystem: an industry that has made irresponsibility its architecture, a scientific community that has dressed engineering benchmarks in the robes of epistemology, and a public — ourselves — that has surrendered critical judgment to the seduction of fluent machines.

I accuse all three. In that order.

I. I Accuse the Machines — Or Rather, Those Who Deploy Them

Let me be precise. The machines themselves are not culpable. They are, as one researcher put it with admirable bluntness, fundamentally “next-token predictors” — probabilistic engines that compress meaning into finite linguistic form. They do not know they are wrong. They cannot know. Wrongness, as an epistemic category, requires a subject who can be held to account. And that is precisely the problem.

The systems deployed at planetary scale today — embedded in medical platforms, legal assistance tools, financial advisory services, educational software, and government interfaces — are designed in such a way that when they fail, no one is accountable. Not the model. Not the company. Not the operator. The terms of service say so explicitly. The architecture enforces it structurally.

Consider what accountability requires. It requires a decision path that can be reconstructed. It requires that the basis for a judgment be identifiable, fixed, and attributable to a specific agent at a specific moment. It requires that the same conditions produce the same result — what philosophers call reproducibility, what engineers call determinism, what courts call evidence.

Current large language models provide none of these things by default.

Ask a deployed LLM why it produced a particular output. It will give you an explanation. That explanation will be fluent, plausible, and internally consistent. It will also be, in a precise technical sense, a confabulation — generated after the fact, bearing no verified causal relationship to the actual computational path that produced the original output. The system does not retrieve its reasoning; it generates a story about reasoning. The log is not the process. The explanation is not the cause.

This is not a bug. It is not a limitation waiting to be fixed in the next release. It is a structural feature of how these systems work — and more damningly, a feature that the industry has shown no urgency to address, because addressing it would slow deployment, increase cost, and — most importantly — make the question of responsibility answerable.

Unanswerable responsibility is commercially convenient.

When a radiologist misses a tumor, there is a name on the report. When an AI-assisted diagnostic system misses the same tumor, the liability evaporates into a fog of shared responsibility distributed among model developers, platform operators, integrating hospitals, and end users — none of whom individually bear sufficient causal weight to be held to account. This is not an accident of legal ambiguity. It is the predictable consequence of deploying systems whose decision processes are, by design, opaque.

I accuse the deployment of AI systems in high-stakes domains without the structural prerequisites for accountability: fixed factual grounding, reconstructible decision paths, reproducible outputs under identical conditions, and explicit human responsibility embedded within the system’s operation — not appended as an afterthought, but built into its architecture as a condition of its running at all.

I accuse the industry of mistaking the absence of a law requiring accountability for the absence of an obligation to provide it.

And I accuse the implicit argument — repeated in boardrooms, keynotes, and regulatory consultations — that the benefits of speed justify the deferral of accountability to some future version that will somehow, through more data and more parameters, become trustworthy by accumulation rather than by design.

It will not. And the people making that argument know it will not.

II. I Accuse the Science — The Myth Dressed as a Law

There is a story that has achieved the status of received wisdom in AI research. It goes like this: intelligence is a function of scale. Given enough parameters, enough data, enough compute, a system will become not merely capable but genuinely intelligent — perhaps even conscious, perhaps even aligned with human values by the sheer pressure of optimization. This story has a name: the scaling hypothesis. And it has been told so many times, by people with such impressive credentials and such spectacular funding, that it has come to feel like a scientific fact.

It is not a scientific fact. It is an engineering observation wearing the clothes of an epistemological law.

The scaling laws documented by Kaplan and colleagues in 2020 are real, within their domain. They describe how model performance on certain benchmarks improves predictably with parameter count, data volume, and compute budget. This is a genuinely useful engineering finding. It tells you how to spend money to improve scores on tests.

It does not tell you what those scores mean. It does not tell you whether the capabilities being measured constitute intelligence in any philosophically defensible sense. It does not tell you whether the system understands anything, or whether it has learned to produce outputs that score well on human-designed evaluations by exploiting statistical regularities in human-generated text — which is a very different thing, and a thing that should not be confused with the first thing, but systematically is.

The benchmark is not the ability. The map is not the territory. The score is not the understanding.

And yet the field has organized itself around benchmark performance as the primary — in practice, often the only — criterion of scientific progress. Models are ranked. Leaderboards are maintained. Press releases announce new state-of-the-art results on evaluations that measure fluency, factual recall under test conditions, and performance on standardized reasoning tasks. What these evaluations do not measure — and what the field has shown striking reluctance to measure — is accountability, reproducibility of reasoning, calibrated uncertainty, and what we might call, without embarrassment, honesty.

Honesty is not a benchmark category.

This omission is not innocent. What you measure is what you build toward. A science that measures performance but not justifiability will produce systems that perform but cannot justify. A science that measures accuracy on test sets but not reproducibility of reasoning will produce systems that are accurate on test sets and unreliable in deployment. We have built what we measured. And we have declined to measure the things that would have made the machines trustworthy rather than impressive.

Now consider the specific case of hallucination — the tendency of language models to generate confident, fluent, and false content. For years, the dominant research paradigm treated hallucination as an error: an imperfection to be reduced through better training, more data, improved alignment techniques. The implicit promise was convergent — more scale, better methods, lower hallucination rates. The implicit prophecy was that sufficiently large, sufficiently well-trained models would eventually approach factual reliability as an asymptote.

This prophecy has not been fulfilled. Hallucination persists across architectures, across scales, across alignment techniques. GPT hallucinates. Claude hallucinates. Gemini hallucinates. The most capable systems available hallucinate with greater fluency and greater confidence than their predecessors, which means their errors are harder to detect and more dangerous to act upon.

Recent theoretical work suggests why this is so, and why it was always going to be so. A geometric analysis of semantic representation — examining how meaning is structured in the high-dimensional spaces of transformer models — finds that hallucination is not a correctable defect but a measurable, lawful consequence of how these systems compress knowledge into finite representational form. The relationship between semantic curvature and hallucination rate is not random noise. It follows a consistent mathematical structure: as the geometry of meaning bends more sharply — at conceptual boundaries, under temporal abstraction, in domains requiring analogical reasoning — the probability of hallucination increases, predictably, independently of how much information the model contains.

This is a profound finding, and not because it is pessimistic. It is profound because it is honest. It replaces the myth of convergent improvement with a principled account of structural constraint. It says: this is not a bug in the implementation. This is a property of the physics of the problem. Finite systems compressing infinite semantic space must curve that space; where it curves, truth bends. The question is not how to eliminate the curvature but how to measure it, predict it, and design around it.

That is science. The scaling hypothesis — in its strong, AGI-implying form — is not science. It is a narrative that serves the interests of those who benefit from the perception that intelligence is a commodity purchasable by the pound of compute.

I accuse the field of dressing that narrative in scientific language.

I accuse the peer review system — journals, conferences, workshop acceptances — of rewarding benchmark improvements while routinely undervaluing theoretical work that questions the foundations of what is being measured.

I accuse the science communication apparatus — press releases, blog posts, breathless technology journalism — of transmitting the scaling myth to the public without the caveats, limitations, and philosophical cautions that would accompany any other scientific claim of this magnitude.

And I accuse, with particular force, the conflation of capability and trustworthiness — the implication, never quite stated but always present, that a system impressive enough to answer your questions fluently is a system trustworthy enough to inform your decisions consequentially.

These are not the same thing. They have never been the same thing. The distinction was known from the beginning and set aside because it was inconvenient.

III. I Accuse Us — The Willing Believers

This is the hardest indictment to write, because it is directed inward.

Human cognition is not well-designed for evaluating the outputs of language models. We know this from decades of cognitive science, and the knowledge has not protected us. We are biased toward fluency. We interpret confident, well-structured language as evidence of competence and reliability — a heuristic that served us reasonably well in a world where fluent speakers were generally more informed than halting ones, but that fails catastrophically when applied to systems that are, in a precise technical sense, optimized to produce fluent outputs regardless of their truth value.

We are susceptible to authority. When a system is introduced as the product of a team of hundreds of researchers at a multi-billion-dollar company, when it is endorsed by prominent academics, when it answers our questions in the vocabulary of expertise, we extend to it the same deference we would extend to a credentialed professional — without recognizing that the credentialing process for AI systems is opaque, self-reported, and governed by the institutions that profit from positive assessments.

We are subject to automation bias — the documented tendency to over-rely on automated systems even when we have reason to doubt them, and to under-apply our own judgment when a machine has rendered one. Automation bias is not laziness. It is a cognitive response to the genuine cognitive cost of critical evaluation. Checking the machine’s work is hard. Trusting the machine is easy. And in a world designed to maximize our engagement with AI outputs, the infrastructure of deference has been deliberately cultivated.

We have, in short, been nudged — by design, by marketing, by the cumulative weight of social proof — into a posture of credulous reception toward systems whose reliability characteristics we do not understand and whose failure modes we are not equipped to recognize.

This is not entirely our fault. But it is our responsibility.

The mistake that Zola identified in the Dreyfus Affair was not that the French public was stupid. It was that they had been given a narrative so emotionally satisfying, so consistent with their existing anxieties and loyalties, that evaluating it critically felt disloyal. The AI narrative offers a similar emotional satisfaction: the promise of tools that amplify human capability, that make hard things easy, that democratize expertise. These are genuine goods, genuinely worth wanting. And they are precisely the goods that make us reluctant to look closely at the conditions under which they are delivered.

I accuse us — the users, the enthusiasts, the early adopters, the policy audiences, the educators, the journalists — of confusing the experience of capability with evidence of trustworthiness.

I accuse us of allowing the emotional appeal of the AI promise to lower our evidential standards.

And I accuse us, above all, of a failure of imagination: the failure to imagine that the tools we find most impressive might be precisely the tools that most require the scrutiny we have been least inclined to apply.

IV. The Evidence That Another Way Is Possible

I said this indictment would not end in despair. It will not.

The charges I have brought are serious. But the point of an accusation is not condemnation — it is correction. And correction is possible, because the alternative already exists. It is not a fantasy. It has been built, and described, and demonstrated. It simply has not been scaled, incentivized, or required.

On accountability: There are architectural approaches to AI systems that make accountability a structural property rather than an external requirement. Systems designed around fixed factual grounding — where the information basis for a judgment is locked before the judgment is made, not retrieved after the fact to support it. Systems with decision traces that record not just outputs but the conditions and constraints under which those outputs were produced, in real time, not reconstructed afterward. Systems in which human judgment is embedded within the decision process as a gating condition — not as a post-hoc review, but as a structural requirement that certain classes of action cannot proceed without explicit human authorization.

These are not science fiction. They are engineering choices. Choices that cost more, deploy more slowly, and make the question of responsibility answerable. The reason they are not standard is not that they are technically infeasible. The reason is that they are commercially inconvenient. Making that choice commercially convenient — through regulation, through liability, through procurement standards — is a policy problem, not a technical one.

On the science: The geometric approach to hallucination described in recent theoretical work represents exactly the kind of honest science the field needs more of. Rather than promising convergence toward reliability through scaling, it asks a harder and more productive question: what are the structural constraints on truth-preservation in any finite system that compresses meaning? The answer — that hallucination is a measurable, lawful consequence of semantic curvature, not a correctable error — is not pessimistic. It is liberating. It transforms an indefinitely deferred problem into a tractable one. You cannot eliminate curvature, but you can measure it, map it, and design systems that monitor it in real time, that flag high-curvature regions as unreliability zones, that adjust their behavior accordingly.

This is what scientific honesty looks like: not the promise of a solution, but the precise characterization of a constraint. Science that describes the geometry of a problem is more valuable — and more trustworthy — than science that promises to make the problem disappear.

The field has the capacity for this kind of work. It needs the incentives, the publication norms, and the cultural values that reward it.

On us: We are not helpless. Cognitive biases are not destiny. Automation bias can be counteracted by training, by interface design, by institutional norms that require human deliberation before AI-assisted decisions become consequential. Susceptibility to fluency can be addressed by education — not AI literacy in the shallow sense of knowing how to prompt a chatbot, but genuine critical literacy: understanding what these systems can and cannot do, what accountability requires, what the difference is between a fluent answer and a justified one.

Civil society has tools. Investigative journalists can follow the money in AI research funding and ask whose interests are served by which findings. Standards bodies can define what accountability requires for AI systems operating in specific high-stakes domains — and those definitions can have teeth, in the form of procurement requirements, insurance conditions, and liability rules. Regulators can require that any AI system used in consequential decisions maintain auditable decision logs, fixed factual grounding, and explicit human accountability at specified points in the workflow.

None of this requires solving the hard problem of consciousness or achieving artificial general intelligence. It requires the political will to insist that the tools we deploy in consequential domains meet the standards we would apply to any other consequential tool.

We require aircraft to have black boxes. We require pharmaceuticals to demonstrate efficacy and safety before approval. We require financial advisors to maintain records of the basis for their recommendations. We do not make these requirements because we expect perfection. We make them because we have decided, as a society, that certain categories of harm require certain categories of accountability, and that accountability requires certain categories of evidence.

We have not yet made that decision for AI. We should. The moment is now — before the systems are so deeply embedded in critical infrastructure that requiring accountability feels like dismantling the infrastructure itself.

V. The Geometry of Responsibility

Let me close with an image drawn from the science I have described.

A semantic manifold — the internal representation of meaning in a language model — is not flat. It curves. Where it curves most sharply, at the boundaries of concepts, in the zones of abstraction and analogy and temporal uncertainty, truth becomes unstable. The model does not know it is entering a danger zone. It proceeds with the same confident fluency it applies everywhere. And it is precisely that confident fluency — the smoothness of the output surface concealing the turbulence of the underlying geometry — that makes it dangerous.

The geometry of irresponsibility works the same way. The surface is smooth. The outputs are impressive. The promises are inspiring. Beneath the surface, in the structure of the system, accountability has been made structurally inaccessible. The decision path cannot be reconstructed. The responsibility cannot be attributed. The error cannot be corrected, because the conditions that produced it cannot be identified.

We are living inside a high-curvature zone of collective reasoning about AI. The confident assertions are everywhere. The fluency is everywhere. The underlying instability — the gap between what these systems actually do and what we are being invited to believe they do — is real, measurable, and consequential.

The correction is not to abandon the technology. It is to insist on the conditions under which technology becomes trustworthy: honest science that measures constraints as well as capabilities, accountable architecture that makes responsibility answerable rather than evaporable, and a public that has decided, with full information, what it is willing to accept.

Zola ended his letter with a challenge: I await.

I will end with a demand, directed at three parties simultaneously:

To the industry: build accountability into the architecture, not into the press release. The tools exist. The choice is yours, until it is made for you.

To the science: measure what matters. Hallucination is geometry, not failure. Trustworthiness is structure, not scale. The honest papers are harder to publish and harder to fund. Write them anyway.

To the rest of us: the fluency of a machine is not a credential. The confidence of an output is not evidence of its truth. We have cognitive biases that make us vulnerable to exactly these confusions, and we have been surrounded by systems optimized to exploit exactly these vulnerabilities. Knowing this is the beginning of the correction.

The geometry of understanding has curvature. Where it curves, truth bends.

The geometry of responsibility has the same property. Where accountability is made structurally inaccessible, trust collapses — not because of any single failure, but because the manifold itself was folded in a way that made failure inevitable and correction impossible.

We can unfold it. But only if we first acknowledge that it was folded.

J’accuse. And I await the correction.

The author draws on research into accountable AI architecture, recent geometric analyses of hallucination in large language models, and two decades of cognitive science on human-automation interaction. The views expressed are the author’s own.