Trust issues with Enterprise AI

For the past few years, we have been teaching AI to speak. The results are impressive. It can summarize documents, draft emails, explain complex topics, and answer questions with remarkable confidence. But once AI enters the enterprise, users eventually ask the question every parent asks a suspiciously confident child:

“How do you know that?”

That question exposes the gap between an impressive demonstration and a dependable AI system. A chatbot can produce an answer. An enterprise AI system must produce an answer grounded in current data, respect access controls, survive partial failures, and provide enough evidence for people to trust it. In other words, enterprise AI has trust issues: confidence is high, but the sources have strong “forwarded many times in the family WhatsApp group” energy.

The first generation of enterprise AI applications often followed a simple pattern:

1. Put a large prompt in front of a language model.
2. Add as much context as possible.
3. Ask the model to produce an answer.
4. Hope everyone is impressed.

This works surprisingly well, until it does not!

Enterprise questions rarely fit neatly inside a single prompt. Consider:

“How is this customer account trending, who owns the open issues, and which opportunities are at risk?”

That is not really one question. It combines customer health, operational ownership, sales opportunities, historical trends, and perhaps several systems of record. Asking one model to solve the entire problem is like hiring one person to be the researcher, database engineer, analyst, security reviewer, and executive communicator. They may be talented, but that is quite a LinkedIn headline, no? The problem is not that the model lacks intelligence. The problem is that the surrounding system lacks structure.

From Chatbot to Agent Team

A more useful design is to treat the AI experience as a team of specialized capabilities.

An orchestrator begins by turning the user’s conversational question into a self-contained request. It resolves references from chat history, identifies the relevant customer or account, and decides which data domains need to participate. In a multi-agent world, specialized skill agents then handle their respective responsibilities. One may query account information, another may inspect support cases, and another may retrieve sales or consumption signals. Their work can happen concurrently before a synthesizer combines the results into one coherent response. The model is no longer pretending to contain the entire enterprise. It is coordinating access to the enterprise. This is an important evolution in AI architecture. We are moving from prompt engineering toward system engineering. Prompts still matter, but so do routing, multi-level decomposition and planning, identity resolution, retries, query validation, telemetry, and evaluation. The clever prompt gets the demo started. The surrounding engineering keeps the product employed.

The NL2Query way: Why decomposition matters

Compound business questions should not become compound headaches.

“How is this customer trending, who owns the open issues, and which opportunities are at risk?”

That single sentence is really three questions wearing a trench coat. Feeding it to one model as one giant prompt tends to produce one giant, unreliable answer. The better approach is to split it up before you try to solve it. A question splitter handles this first. It is its own dedicated model call, separate from the one that eventually writes the answer, and its only job is to break a compound question into smaller, independently answerable sub-questions. Since LLM output formatting is never fully guaranteed, the parser reading that output has to be forgiving — it needs to handle a clean JSON array, a quoted comma-separated list, or something more loosely formatted, and it needs a sensible fallback. If the decomposition itself cannot be parsed cleanly, the system simply treats the original question as a single unit rather than failing outright. Decomposition should make things safer, never riskier. Each resulting sub-question is then turned into an executable query — SQL for a relational store, DAX for a semantic model — using a prompt built from three layers stacked together: a description of the data model, business rules that define what terms like “open” or “at risk” actually mean in this data, and a small set of few-shot examples pulled from a domain-specific registry to match the shape of the question being asked.

A few implementation choices make this fast and cheap rather than just functional:

• A prompt-cache boundary. The large, static part of the prompt — schema, rules, examples — is deliberately separated from the small, per-sub-question part using a fixed marker. Since that static portion is identical across every parallel call, the model provider’s cache can be hit again and again instead of reprocessing the same context from scratch every time.
• Seeded, deterministic decoding. Query generation runs with a fixed decoding seed, which means the same question tends to produce the same query. That is valuable for debugging, for regression testing, and for getting more cache hits on repeated or similar questions.
• A staggered head start. The first call in a batch is sent slightly ahead of the rest, so it can warm the shared prompt cache before the remaining parallel calls arrive right behind it.
• A hard ceiling on output length, so a malformed or runaway generation cannot quietly balloon latency or cost.
• A read-only gate before execution. Every generated query passes through an allow-list check before it touches real data. Read-only statements pass. Anything that writes, deletes, or alters is blocked outright — regardless of what the model intended.

Put this all together and a few real advantages show up:

• Smaller tasks are just easier to get right. A model generating one focused query has a much narrower way to fail than a model trying to answer everything at once.
• Independent work becomes parallel work. Sub-questions run concurrently instead of one after another.
• A single failure stays a single failure. If one sub-question’s query breaks, the rest of the answer can still come back intact, clearly labeled.
• Every claim is traceable. Each part of the final answer maps back to a specific sub-question, a specific generated query, and a specific result — which is what makes the answer auditable instead of just convincing.
• New domains slot in without surgery. Adding a new area of the business means adding a new schema description, rule set, and example set to the registry — not rewriting the splitter or the orchestration logic.

It is less “one model to rule them all” and more “the right model call, with the right cached context, for the right job.” That is a healthier way to delegate than betting everything on a single know-it-all model !

The Graph way: Sometimes the Answer Is in the Relationships

Specialized agents help determine where to look. But enterprise intelligence also depends on understanding how things are connected. Traditional retrieval systems usually return documents or text chunks. That is useful when the answer lives in a paragraph. Many business questions, however, are not document questions. They are relationship questions.

Who owns this account? Which cases affect its health? Which opportunities involve the same product? How are recent consumption changes connected to customer risk?

This is where knowledge graphs and GraphRAG become particularly valuable. In a customer graph, accounts, contacts, cases, products, opportunities, owners, and health signals become nodes. Their relationships become edges. Instead of retrieving isolated text, the system retrieves a focused subgraph containing the entities and connections most relevant to the question. The AI receives not only facts, but also context about how those facts relate. You could say this gives retrieval an ‘edge’ (pun intended). Several edges, ideally!

GraphRAG introduces a useful shift in how we think about grounding.

Classic RAG asks: “Which pieces of text resemble this question?”

GraphRAG can ask: “Which entities are relevant, and what connected evidence surrounds them?”

A query is embedded into the same vector space as the graph’s nodes. The most semantically relevant nodes become starting points. The system then expands outward dynamically to collect nearby context before passing the resulting subgraph to the language model.

But graph expansion needs boundaries. Some nodes are extremely well connected. A region, product, sales stage, or account owner may link to hundreds of records. Traversing through every high-degree node can turn a focused investigation into an all-company meeting. Practical GraphRAG therefore needs controls such as:

• A small set of highly relevant starting nodes
• A limited number of expansion hops
• Blocking traversal through large hub nodes
• Caps on records and total nodes
• Ranking and truncation when the graph exceeds its context budget

The goal is not to give the model the whole graph. The goal is to give it the right isolated subgraph.

Trust Is a Product Feature

Grounding an answer in live data is necessary, but it is not sufficient. A trustworthy system must also govern what the AI is allowed to do. If an agent generates SQL or another analytical query, that query should pass through explicit controls before execution. Read-only operations can be allowed while destructive statements are blocked. Result sizes can be capped. Authentication can use managed identity rather than embedded credentials. Row-level security is to be ensured for the data access. The system should assume generated code is untrusted until validated. The same principle applies to visualization. When a GraphRAG system returns the exact nodes and relationships used to produce an answer, the graph becomes more than eye candy. It becomes an evidence surface. Users can inspect what the model saw. Developers can diagnose weak retrieval. Reviewers can determine whether an important relationship was omitted. Instead of asking users to trust a polished paragraph, we let them examine the source of truth.

Resilience Is Part of Intelligence

An AI system can fail in many creative ways. A model endpoint may become rate-limited. A downstream skill may time out. An entity may fail to resolve. A graph artifact may be unavailable. A follow-up-question generator may decide that valid JSON is merely a suggestion. Production systems need to expect these failures. That means adding retries with backoff, secondary endpoints, per-skill error isolation, fallback retrieval modes, and graceful degradation for non-essential features. If semantic graph retrieval fails, keyword retrieval may still provide useful results. If follow-up generation fails, the primary answer should still succeed. If one skill cannot respond, the system can return evidence from the remaining skills while clearly communicating the limitation.

The important lesson is that reliability does not sit outside AI quality. It is part of AI quality. A brilliant answer that arrives inconsistently is not an intelligent product. It is a recurring calendar surprise.

Evaluation Cannot Be a Final Exam

One of the largest shifts for AI leaders is recognizing that evaluation is not a one-time release gate. Traditional software can often be tested against deterministic outputs. Generative systems require a richer approach because a response can be phrased differently while remaining correct, or sound excellent while being factually wrong.

A mature evaluation framework can combine:

• Curated benchmark questions
• Ground-truth answers and expected metrics
• Semantic similarity scoring
• Required facts and response-format checks
• LLM-as-judge assessments
• Development-versus-production comparisons
• Latency and failure-rate monitoring
• Concurrent burst tests

This evaluation should run continuously. When a prompt changes, a model is upgraded, a data schema evolves, or a business rule is added, the benchmark suite should reveal whether the system improved, regressed, or merely became more eloquent. Models may be nondeterministic, but our quality discipline should not be. Or, put differently: never let your AI grade its own homework without a rubric, a proctor, and a suspiciously detailed audit log.

What AI Leaders Should Optimize For

The next phase of enterprise AI leadership will require a different scorecard.

1. Evidence over eloquence — A fluent answer is useful. A fluent answer connected to live, governed evidence is deployable.
2. Coordination over omniscience — Do not force one prompt or model to perform every task. Use orchestration, specialized skills, and deterministic components where they fit best.
3. Inspection over illusion — Expose retrieved evidence, relevant relationships, execution status, and limitations. Confidence should come from traceability, not typography.
4. Graceful degradation over artificial perfection — Design for partial failure. The system should preserve useful work and communicate boundaries instead of collapsing completely.
5. Continuous evaluation over launch-day excitement — A successful demo proves that an AI system worked once. Continuous evaluation helps prove that it continues to work.
6. Architecture over model obsession — The newest model may improve performance, but model selection is only one part of the system. Retrieval quality, data governance, latency, identity, orchestration, and observability often determine whether the product succeeds.
7. The Next Interface Is Inspectable Intelligence

The chatbot gave us an answer.

The AI system gives us the answer, the evidence, and the investigative team behind it.

Nothing teaches AI system design like production. I learned that the model is the easy 20 percent; grounding, access control, failure recovery, and evaluation are the other 80 percent nobody demos. The chatbot nailed the easy 20 percent. The Enterprise grade AI system had to earn the other 80 — the evidence, the guardrails, and the recovery plan behind every answer.

Now that is what I call an agent of change!