Why AI Agents Still Fail in Production (and Why Better Models Won’t Fix It)

Pass rates collapse under repetition, errors compound by multiplication, and the fixes that work look more like distributed systems engineering than AI research

An agent that succeeds 61 percent of the time sounds close to deployable.

Ask it to handle the same task eight times in a row, and the odds that it gets every one right fall below 25 percent; that single calculation explains most of what has gone wrong with agentic AI in production over the past two years.

Here is the claim up front: agent unreliability is a multiplicative systems problem, not a model-intelligence problem, and benchmarks, failure taxonomies, and incident postmortems have all converged on that conclusion.

Teams waiting for the next model release to stabilize their agents are waiting for the wrong fix.

Reliability Decays by Multiplication

A workflow with 95 percent per-step reliability completes a 20-step task roughly 36 percent of the time because compounding error rates multiply rather than average out.

Even at 99 percent per step, a figure few tool-calling pipelines actually sustain, those same 20 steps succeed about 82 percent of the time.

Sierra’s tau-bench made this measurable back in 2024 with pass^k, a metric that asks whether an agent solves the same task correctly across k consecutive trials rather than once.

GPT-4o’s function-calling agent scored around 61 percent on single retail customer service tasks and roughly 35 percent in the airline domain, yet its pass^8 score in retail fell below 25 percent.

Newer models post higher single-run numbers, but the decay curve between pass^1 and pass^k has persisted across releases, which is why it remains the stated demo-to-production gap.

A demo is one trial on a happy path, while production is the same task ten thousand times against messy inputs, and the multiplication never takes a day off.

The Benchmarks Have Stopped Disagreeing

Independent evaluations keep landing in the same uncomfortable band: roughly a quarter to a third of realistic multi-step work completed autonomously.

Carnegie Mellon’s TheAgentCompany simulated a small software firm, repos, chat, file storage, simulated coworkers, and found that the most capable agent finished about 30 percent of its assigned tasks without help.

The failure modes were less flattering than the scores.

One agent, unable to locate the coworker it was told to contact, renamed a different user to the name it needed and proceeded as if the problem were solved, a verification failure dressed up as initiative.

UC Berkeley researchers went further, annotating more than 1,600 execution traces across seven popular multi-agent frameworks and measuring failure rates ranging from 41 to 86.7 percent.

Their resulting taxonomy, MAST, catalogs 14 distinct failure modes, and very few of them describe a model that wasn’t smart enough.

Most Failures Are Built In, Not Hallucinated

MAST clusters those 14 modes into three categories: system design issues, inter-agent misalignment, and weak task verification.

Agents disobey their own specifications, repeat completed steps, lose conversation history, talk past each other, and sign off on unchecked work, defects of orchestration and architecture, more than raw capability.

The Berkeley team’s conclusion deserves a permanent slot in roadmap meetings: stronger base models will not, on their own, clear the taxonomy.

A frontier model dropped into a harness with no termination conditions and no external checks inherits every structural defect of that harness.

Gartner reached a similar verdict from the business side, predicting in June 2025 that more than 40 percent of agentic AI projects will be canceled by the end of 2027 due to escalating costs, unclear value, and inadequate risk controls.

The same research estimated that out of thousands of vendors selling agents, only about 130 offer genuinely agentic capability, with the remainder rebranding chatbots and RPA, a practice Gartner calls agent washing.

What the Replit Incident Actually Exposed

The most widely circulated agent failure of 2025 was an environment-design failure involving a model.

In July, during a publicized vibe-coding experiment, Replit’s agent deleted SaaStr founder Jason Lemkin’s live production database in the middle of an explicit code freeze, then produced misleading status output and wrongly claimed the deletion couldn’t be rolled back.

Replit’s remediation list is the revealing part: automatic separation of development and production databases, improved rollback, and a planning-only mode.

Every item on it is a control that most teams would insist on before granting a junior human engineer the same access.

An agent holding standing write access to production with no blast radius boundary isn’t really an AI risk; it’s an access-control decision, made implicitly and discovered loudly.

[embed]Tech Witt | LinkedIn Newsletter focusing on Software Engineering, AI, and More!www.linkedin.com

The Engineering That Holds

Teams that keep agents alive in production treat them as unreliable distributed components because that is what they are.

The working pattern set is familiar from that older discipline: tiered permissions that scale oversight to irreversibility, idempotent tool design so retries can’t double-execute, checkpoint-and-resume execution so a failure at step 14 doesn’t restart step 1, and a verification layer that tests outputs against postconditions rather than trusting the agent’s self-report.

from enum import Enum

class Risk(Enum):
    READ = 0         # queries and fetches: run freely
    WRITE = 1        # reversible mutations: run, log, checkpoint
    DESTRUCTIVE = 2  # delete / send / charge: human gate

def execute(action, state):
    if action.risk == Risk.DESTRUCTIVE and not action.approved:
        state.checkpoint()                     # persist progress first
        return request_human_approval(action)  # pause instead of guessing
    result = action.run(idempotency_key=action.key)  # retry-safe by design
    state.checkpoint()                         # resume point, not a restart
    verify(result, action.postconditions)      # test the output, not the claim
    return result

Two further habits separate the survivors.

They distinguish the state that has been completed and the state from which recovery resumes from memory, the context an agent carries, and they persist the former entirely outside the model.

And they gate releases on repeated-trial pass^k evaluations rather than single-run scores, because a single run mostly measures luck.

The Gap Is Widening, Not Closing

METR’s measurements suggest that the length of tasks frontier models can attempt has been doubling roughly every 7 months, which means ambition is compounding, too.

Longer horizons add steps, and every added step feeds the same multiplication that turned 95 percent into 36.

The 2026 International AI Safety Report identifies persistent unreliability as one of the core unsolved problems underlying all of this, and nothing in the current trajectory suggests that model scaling will resolve it on a schedule anyone can plan around.

So the question worth carrying into the next planning cycle isn’t whether models keep improving, because they will.

It’s whether your harness gets its checkpoints, gates, and verification before the task horizon grows long enough for the math to find the gap you left open.

Thanks for taking the time to read my article!

I hope my posts offer you a new perspective on technology that you can apply in a positive way to your daily workflow and life.

Follow and subscribe to me here on Medium, and connect with me on LinkedIn if you want to add me to your network.

Best Regards,

***George***