Why AI Agents Break in Production

Demos are short. Production is long.

AI agent demos compress the world into a clean, forgiving five minutes. The task is tidy. The tools behave. The context is fresh. The failure case gets edited out. Enterprise production is the opposite. Inputs are messy, systems are inconsistent, policies conflict, and the same workflow has to work on Tuesday afternoon exactly as well as it did in the sales demo.

That is why so many agent products look further along in public than they do inside real companies. The technology can already be impressive. The operating discipline around it is still immature. Production asks harder questions than a benchmark ever will. What happens on the fourth retry. What happens when the tool response is partial. Who sees the trace. Who approves the action. How do you know the agent failed for the right reason instead of succeeding for the wrong one.

The gap is not between intelligence and stupidity. It is between something that can occasionally complete a complex task and something that can do it repeatedly, with an audit trail, and under real business constraints. That is the reliability gap this market keeps running into.

AI use is broad. Scaled value is still rare.

It is important to describe the market accurately. Enterprise AI adoption is not low in the sense of experimentation or basic usage. McKinsey's November 5, 2025 global survey found that 88% of respondents say their organizations regularly use AI in at least one business function. But only about one-third say their companies have begun scaling AI programs across the enterprise. Twenty-three percent say they are scaling agentic AI somewhere, and in any given business function no more than 10% say agents are scaled. Experimentation is broad. Scaled deployment is still narrow. (1)

That same pattern shows up in value capture. McKinsey found that just 39% of respondents attribute any enterprise-level EBIT impact to AI, and most of those say the impact is less than 5%. BCG's 2025 research is even blunter. Only 5% of companies qualify as value-generating future-built leaders, while 60% report hardly any material value from AI so far. The story is not that nobody cares about AI. The story is that most organizations still have not translated enthusiasm into durable operating results. (1)(2)

Agents sit right in the middle of that bottleneck. They are attractive because they promise to move from answering questions to actually doing work. But the minute an agent leaves the sandbox of a controlled demo and touches production systems, the bar changes. Accuracy is not enough. Reliability, recovery, cost discipline, and auditability start dominating the conversation.

Benchmarks are useful. Reliability is what gets bought.

Benchmarks still matter. They tell you whether models can reason, follow instructions, use tools, and improve over time. Ignoring them would be foolish. Mistaking them for production readiness is worse.

METR's July 10, 2025 study on experienced open-source developers is a useful correction. In a realistic setting, developers working on repositories they already knew took 19% longer when using early-2025 AI tools. The researchers explicitly caution that benchmark tasks are often self-contained, algorithmically scored, and cleaner than real work. Those properties can overstate real-world usefulness. (3)

METR's work on task time horizons makes the same point from another angle. A model's time horizon is a reliability curve, not a victory lap. Their framing is blunt. A two-hour horizon does not mean the model can autonomously perform all two-hour knowledge work. It means success drops as tasks get longer, messier, and more dependent on real context. When evaluation moves closer to holistic, real-world judgment instead of tidy algorithmic scoring, performance also gets worse. Enterprise workflows are usually closer to the messy side. (4)

In production, the relevant metric is not whether an agent can sometimes finish a long task. It is whether it can do the right subset of tasks repeatedly with bounded failure modes. Enterprises buy confidence. They buy logs, traces, review queues, and the ability to say no. A system that dazzles on a leaderboard but cannot explain a bad action is not enterprise-ready.

Architecture matters, but there is no perfect recipe.

This is where architecture discussions become both useful and overrated. Useful, because architecture really does change reliability, cost, and failure behavior. Overrated, because there is no universal winning pattern. The right architecture is the one whose complexity is justified by the workflow in front of it, not the one with the most boxes in a diagram.

OpenAI's practical guide recommends maximizing a single agent's capabilities before splitting work across multiple agents. Anthropic makes a similar argument from the enterprise side: start simple, scale intelligently, and only pay for complexity that earns its keep. That advice is less glamorous than the latest orchestration framework. It is also much closer to how reliable systems usually get built. (5)(6)

Orchestrator systems help when a problem naturally decomposes. Research, multi-source synthesis, and specialized tool domains often benefit. But they also introduce new failure classes: context dilution at the coordinator, conflicting subagent outputs, retry storms, and higher token spend. Anthropic's own architecture guide warns that observability becomes more critical as agent decisions multiply. Without tracing across delegation and synthesis, debugging quickly becomes guesswork. (6)

Sandboxes and code execution add a different trade-off. They can dramatically increase capability because the agent can test, inspect, and act rather than just speculate. But they also expand the blast radius. Once agents can touch files, APIs, or production systems, enterprise teams stop asking whether the demo worked and start asking whether access controls, rollback paths, and audit logs are real. That is the right question.

Why teams overbuild.

The industry still has a habit of reaching for architecture before it has exhausted simpler levers. A fuzzy task becomes a router. A weak prompt becomes a hierarchy. An unclear tool contract becomes another specialist. Sometimes that is justified. Often it is just complexity laundering.

A surprising number of failures that get blamed on model limitations are really failures of scope, tool design, or operational feedback. The agent was asked to do too much in one run. The tool description was ambiguous. The output contract was loose. There was no eval set tied to the real workflow. There was no trace that showed where the decision went off course. Adding more agents on top of that foundation usually multiplies the confusion instead of fixing it.

This is one reason hype cycles distort engineering judgment. New frameworks, deeper orchestration, and more autonomous loops look like progress because they are easy to demo. Better prompt discipline, narrower workflows, cleaner tools, and stronger evals look mundane. In production, the mundane work tends to win.

What production teams should optimize for instead.

NIST's generative AI profile frames the real agenda more soberly around trustworthiness, measurement, evaluation, and risk management. OpenAI's own production support write-up says much the same thing in practice. The primitives that mattered were step-level traces, classifiers, evals, and feedback loops that continuously improve the system. Not just one more clever chain of reasoning. (7)(8)

If you want agents to survive enterprise production, start with narrower tasks and sharper contracts. Make tools explicit. Give the model fewer degrees of freedom. Define stop conditions. Add human review where the downside justifies it. Build evals from actual failures, not imagined ones. Trace every meaningful step. If the team cannot explain why the agent made a decision, it does not yet control the system.

The best production agent stacks often look less like autonomous coworkers and more like disciplined operators. They know when to ask. They know when to stop. They know what data they are allowed to see. They expose their work. They fail in ways that are legible to humans. That is not a retreat from ambition. It is what ambition looks like after contact with reality.

Reliability also has to include economics. Multi-agent systems, deep loops, and repeated tool retries all compound cost. Anthropic's framework makes the trade-off clear: as architectures become more complex, the added token spend and coordination overhead have to be justified by business value. If the architecture is too expensive to run at the frequency the workflow demands, it is not production-ready no matter how sophisticated it looks. (6)

Less surface area, more trust.

The broader lesson is simple. Less surface area often creates more trust. Fewer moving parts make it easier to observe the system, test it, reason about failure, and improve the details that matter. Enterprises do not need the most agentic possible architecture. They need the most dependable architecture that solves the job.

That is also why there is no perfect recipe. BCG's language is the right one here: each company will find its own path. The path depends on workflow shape, risk tolerance, integration depth, and what counts as failure in that environment. A legal review agent, a support agent, and an internal research agent should not be built to the same operating model just because they all get labeled agents. (2)

Our own bias is straightforward. If an agent cannot show its work, operate within clear boundaries, and fit the workflow it is entering, it is not ready for enterprise production. That is why we keep coming back to observability, targeted actions, and systems that stay simpler for longer. Reliability is not a feature you bolt on after the demo. It is the product.