Agentic AI Hype Cycle: What’s Real vs. What’s Missing

The demo runs perfectly. The agent receives a request, decomposes it into subtasks, calls tools, handles errors, and produces a result. The audience is impressed. Then someone asks how it performs in production and the room goes quiet.

The gap between the agentic AI demo and the agentic AI deployment is the most important story in applied AI right now, and it is not being told clearly. The hype says agents will replace knowledge workers. The reality says most production agent deployments today handle narrow, bounded, well-defined tasks — and the teams that have shipped agents at serious scale have paid a steep engineering price to get there.

This is not a pessimistic article. Agents are genuinely useful for a specific and expanding category of work. But the work that is real, the work that is overhyped, and the infrastructure that is still missing are three different things, and conflating them produces bad architecture decisions.

Disclaimer: The opinions expressed in this article are my own and do not represent the views of Google. This content is based solely on publicly available information.

What Agents Are Actually Good At Today

The categories where agents consistently deliver value in production share two properties: the output is verifiable and mistakes are reversible.

Code generation and refactoring. An agent that writes a function, runs the tests, reads the failures, and iterates is genuinely more capable than a one-shot LLM call for many coding tasks. The output is verifiable (tests pass or they do not), the cost of a bad output is low (revert the diff), and the iteration loop is tight. This is why coding agents — GitHub Copilot Workspace, Cursor Agent, Claude Code — are the most successful deployed category by a wide margin. Internal studies from Anthropic and Google published in 2025 showed coding agent tasks completing successfully 60–75% of the time in well-defined repositories, compared to under 20% for general open-ended tasks.

Data transformation pipelines. An agent that reads a schema, writes a transformation, validates the output against constraints, and retries on violation is solving a real problem: the schema matching and constraint validation that make ETL brittle. Output verifiability (schema validation, row count checks, spot samples) keeps the agent in a safe loop. Teams running these pipelines at scale report 40–60% reduction in manual data-engineering intervention on routine transformation work.

Structured document extraction. An agent that processes an invoice, extracts fields, flags uncertainty, and routes low-confidence items to a human review queue handles the Pareto tail of documents that pure structured extraction misses. The “human in the loop for low confidence” pattern converts the hard reliability problem into a throughput problem. At an accuracy threshold of 90% confidence, agents handle roughly 70% of documents automatically; the remaining 30% go to humans at reduced cost because the agent has pre-filled most fields.

Retrieval-augmented question answering with citations. Not the open-ended “chat with your documents” demo — the narrow enterprise version: given a defined corpus, a constrained question type, and required citation of source passages, agents can achieve production-grade accuracy. The citation requirement is load-bearing — it makes the output verifiable by a human without requiring the human to repeat the research.

What all of these share: the agent operates in a loop where each step has a deterministic success signal. The agent is not judging whether its output is good — the environment tells it.

The Reliability Problem: Why Multi-Step Failure Compounds

The failure mode that undoes most ambitious agent deployments is not any single step failing — it is failure compounding across steps.

If each step in an agent pipeline succeeds with probability p, an n-step pipeline succeeds with probability p^n. The math is merciless. Let’s look at this math in below simple test about above math. As shown in Figure 1, the right panel makes the operational implication concrete: at 95% per-step reliability — which sounds strong — the pipeline falls below 80% end-to-end success by the fifth step.

Figure 1: End-to-end agent success rate as a function of pipeline length. The right panel shows how quickly reliability collapses — at 95% per step, you fall below 80% end-to-end success by the fifth step. This is why the demos look good and production deployments are hard: the demo is 3–5 steps, the production task is 15–30.

This is the central engineering tension in agentic AI. The per-step model quality looks fine on every benchmark. The multi-step pipeline behavior only appears at production scale.

The mitigation strategies are real but each carries a cost:

• Short pipelines: keep agent tasks to 3–8 steps maximum. This is a design constraint that forces you to define “task” differently than you would for a human.
• Deterministic steps: replace probabilistic LLM calls with deterministic tool calls wherever possible. Code execution, database queries, schema validation, and API calls have p ≈ 1.0 — use them to anchor the pipeline.
• Verification at checkpoints: insert environment-based verification after every 2–3 steps. The agent checks its output against a ground truth signal, not against its own judgment.
• Retry budgets: plan for retries explicitly. A 3-retry budget on a 90%-per-step task improves per-step success to 1 - (1-0.9)^4 ≈ 99.99% — but triples the cost and latency. Use reliability.success_with_retry(0.90, 3) to model this for any p.

The Economic Reality: Cost Grows Superlinearly

Agent cost is not a flat multiplier on the underlying model API cost. It grows superlinearly with pipeline length because of two compounding factors: scaffolding overhead (system prompts, tool definitions injected on every step) and context growth (each step appends its output to the running conversation history).

A direct API call costs tokens for the prompt plus output. An agent pipeline with n steps costs the initial prompt, plus n copies of the tool definitions, plus the accumulated context from all prior steps, plus each intermediate response. The growth is superlinear in n. Let’s check this math with simple test.

As shown in Figure 2, the cost curve bends upward sharply past 8 steps. A 15-step agent on Sonnet 4.6 costs roughly 8× more than a direct API call at the same query, not because each LLM step is expensive in isolation, but because the accumulated context and repeated scaffolding drive input token counts up at every step.

Figure 2: Agent economics on Claude Sonnet 4.6. Left: pipeline cost grows superlinearly from $0.003 (1 step) to $0.026 (15 steps) while the direct-call baseline stays flat at $0.0076. Right: the ROI curve — value delivered relative to cost — peaks around 5–6 steps for most current task categories and declines as reliability problems consume the value gains from additional steps.

The practical implication: agents have a cost-performance sweet spot in the 3–8 step range for most current tasks. Below that, a well-engineered direct call is faster and cheaper. Above that, the reliability and cost math turns against you unless you have verification infrastructure to keep each step deterministic.

The Missing Infrastructure

The production problems that go undiscussed in most agentic AI coverage are infrastructure problems, not model problems. The model will improve. The infrastructure is each team’s responsibility — and it largely does not exist yet as a default.

Agent observability: What did the agent actually do? Which tool calls happened, in what order, with what arguments, and what did they return? How long did each step take? Where did the agent deviate from the intended plan? In a conventional service, this is solved by structured logging, distributed tracing, and dashboards. For agents, it requires instrumentation at the LLM call level, the tool call level, and the step boundary level simultaneously — and most teams build this from scratch for every agent because there is no standard telemetry format for agent behaviour. Teams that have added step-level tracing universally report that the logs revealed failure modes they did not know existed — tool calls silently returning empty results, intermediate plans diverging from user intent at step 2, context window overflows causing silent truncation of tool definitions.

Agent testing: How do you write a test for a non-deterministic pipeline? The answer most teams settle on is “run it end-to-end on a set of representative inputs and evaluate the output.” This is slow, expensive, and catches only the failure modes you thought to test for. Unit testing individual tools is tractable; testing the agent’s decision-making is not. Which prompts trigger which planning paths? Does the agent handle edge cases in tool outputs? Do model version upgrades change multi-step behaviour in ways that aren’t visible in single-step benchmarks? The state of the art is “eval harnesses with LLM judges or human raters,” which does not scale to continuous integration cadences.

Rollback and recovery: When an agent fails midway through a task that has side effects — it has sent an email, committed a change, or called a write API — what happens? Can the operation be compensated? Who is notified? The infrastructure for saga-style compensating transactions exists in distributed systems engineering and is nearly absent from agent frameworks. Teams that have shipped agents with external side effects have built this themselves, and the effort is typically reported as taking 2–4 weeks for a minimal safe implementation.

Cost monitoring per task: Agent pipelines are expensive in proportion to their complexity, and cost is hard to attribute at the task level. A request that triggers a 20-step pipeline costs an order of magnitude more than one that completes in 3 steps, but both appear as “one user query” in standard API cost dashboards. Without per-task cost attribution, teams cannot detect runaway agents, optimise pipeline length, or set cost budgets that have any meaning.

Where the Hype Overshoots

The infrastructure gap is real, but it is also misread. Vendor marketing and conference demos have inflated expectations far beyond what the reliability and cost math actually supports. Three specific claims deserve scrutiny.

“Agents will replace knowledge workers.” This conflates “agents can do individual tasks that knowledge workers do” with “agents can replace the judgment, context, and accountability that knowledge workers provide.” The tasks agents handle well today — code writing with test verification, invoice field extraction, document search with citation — are narrowly-defined, repeatedly-performed, and outcome-verifiable. Knowledge work consists largely of tasks where the problem definition is itself ambiguous, output quality is subjective, and the consequence of error propagates invisibly. That is not an infrastructure problem — it is a fundamentally different category of task.

“Multi-agent systems multiply capability.” A multi-agent system multiplies the compound reliability problem. If a single agent at 95% per-step succeeds 60% of the time on a 10-step task, and you build an orchestrator that delegates to three such agents and synthesises their outputs, you are stacking three 60%-success pipelines behind a fourth pipeline that must combine them coherently. The teams that have shipped multi-agent systems at production scale are rare, and none of them make it sound easy. The complexity is not additive — it is combinatorial.

“The agent just needs a better model.” Model quality helps. Per-step reliability improving from 90% to 95% roughly doubles the number of steps you can take before success rates collapse below 80%. But better models do not fix absent observability, missing rollback infrastructure, or superlinear cost growth. They raise the ceiling; they do not replace the engineering required to operate below it.

“Autonomous agents are production-ready.” The benchmark is not the right frame. GAIA, SWE-bench, and similar agent evaluation frameworks measure best-case performance on well-defined tasks with known ground truth. Production performance — on ambiguous inputs, with partially-broken tools, under cost and latency constraints, against a continuously changing environment — is systematically worse than benchmark numbers suggest. Teams that quote SWE-bench numbers when pitching agent deployments are not being dishonest; they are measuring the wrong thing.

The Actual Near-Term Trajectory

The agent category that will grow fastest in the next 18–24 months is not the general-purpose autonomous agent — it is the narrow, deeply-integrated, workflow-specific agent with explicit human checkpoints.

The pattern looks like this: a specific business process that currently requires human judgment at 3–5 decision points is reimplemented as an agent pipeline. Each decision point becomes either an automated tool call (if the decision is deterministic) or a human review step (if it is not). The agent handles everything between checkpoints. The human handles the checkpoints. Total time and cost go down substantially; reliability is preserved because the hard decisions still go to a human.

This is less exciting than the demo. It is also what is actually shipping and generating real ROI. The infrastructure investment worth making now:

1. Structured agent logging with step-level trace IDs that connect LLM calls, tool calls, and step outcomes in a single queryable trace. This is the minimum viable observability requirement for debugging in production.
2. Evaluation harnesses for agent-specific failure modes: tool call argument validity, step sequence correctness, output verification against ground truth on a sample of production traffic. Run these in CI on every model version bump.
3. Cost attribution by task so you can identify which request types are generating 80% of agent API spend. A per-request cost breakdown, not just an aggregate, is the only way to find runaway pipelines before they become incidents.
4. Rollback primitives for every side-effectful tool. If the agent can send an email, there should be a dry-run mode, a confirmation step, and a record of what was sent. “The agent did something irreversible and we don’t know what” is not an acceptable production state.

The teams that invest in this infrastructure now will be the ones running production agents at scale in two years. The teams that don’t will be demoing the same impressive demo while the agents continue to fail on the third step.

Key Takeaways

Putting it all together: the argument is not that agents are hype, but that the claims and the math need to match. Here is what the evidence actually supports.

1. Agents work today for bounded, verifiable, reversible tasks. Code generation with test execution, structured document extraction with validation, and narrow question-answering with citation have crossed the threshold into production reliability. Open-ended, long-horizon, side-effect-heavy tasks have not.
2. The compound reliability problem is not a model problem. A 10-step pipeline at 95% per-step success delivers end-to-end success only 60% of the time. Better models raise the ceiling; they do not change the math. Short pipelines with deterministic checkpoints are the design pattern that actually works.
3. Cost grows superlinearly with pipeline length. Scaffolding overhead and context accumulation push a 15-step agent to 8× the cost of a direct API call. The cost-performance sweet spot is 3–8 steps for current models.
4. The missing infrastructure is your problem, not the model provider’s. Agent observability, testability, rollback, and cost attribution are not coming pre-built. Teams that have shipped production agents have built these themselves. Budget for it.
5. The near-term winner is the narrow, deeply-integrated, checkpoint-driven agent. General-purpose autonomous agents are a research target. Narrow workflow-specific agents with human checkpoints at the hard decisions are what generates ROI in 2025–2026.

The hype says agents are here. The math says agents are here for the right tasks, with the right infrastructure, at the right pipeline length. That is a smaller and more specific claim — and it is the one worth engineering toward.

Thank you for reading my post, and I hope it was useful for you. If you enjoyed the article and would like to show your support, please consider taking the following actions:

👏 Give the story a round of applause (clap) to help it gain visibility.

📖 Follow me on Medium to access more of the content on my profile. Follow Now

🔔 Subscribe to the newsletter to not miss my latest posts: Subscribe Now or become a referred Medium member.

🛎 Connect with me on LinkedIn for updates.