Why Your AI Project Will Fail at the Data Layer

More than 80% of AI projects fail. The model is almost never the reason. Here’s how to read an AI development quote before you sign it.

You scoped an AI agent. You picked a model. You got three quotes back, and the cheapest one looked clean: a model build, an integration line, a four-week timeline. You’re about to sign it.

Stop for a second.

The model is not where this project fails. The data is. And the quote in front of you almost certainly doesn’t say so.

This is the part of AI development nobody demos. It’s also the part that decides whether you ship.

The number that should change how you scope

RAND Corporation, the independent research organization, studied this directly. In their 2024 report, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed, researchers interviewed 65 experienced data scientists and engineers and found that more than 80% of AI projects fail. That’s twice the failure rate of IT projects that don’t involve AI.

MIT’s NANDA initiative found something similar from a different angle. Their 2025 report, The GenAI Divide: State of AI in Business, analyzed 300 public AI deployments and found that roughly 95% of generative AI pilots delivered no measurable impact on profit and loss. Only about 5% produced real revenue acceleration.

And it’s getting worse, not better. S&P Global Market Intelligence found that 42% of companies abandoned most of their AI initiatives in 2025, up from 17% the year before.

Here’s the part that matters for you. RAND’s interviewees named data problems as one of the leading root causes of failure, second only to leadership and misaligned goals. One of them put it bluntly: 80% of AI work is the unglamorous labor of data engineering, and weak people doing that work poison everything downstream.

So the failure rate is real, it’s well-documented, and a large share of it traces back to one place. The data layer. The exact thing missing from the quote on your desk.

Why the data problem is invisible until it’s expensive

The reason data gets skipped in proposals is simple. It’s hard to see before you start, and easy to wave away.

A vendor reads your brief. The brief says “an AI agent that processes our invoices.” The vendor pictures clean invoice records in a tidy database. They price the model, price one integration, add a timeline, and send it over. Fast quote, low number, easy yes.

Then they see your actual data.

The invoices live across 12 email inboxes. Three different PDF layouts, none of them consistent. A legacy ERP where the same vendor is coded four different ways. Half the historical records are missing a field the model needs. Now the project the vendor scoped does not exist anymore. A different, larger project has taken its place.

This is why quotes for the same brief vary by 10x. We’ve seen the same one-line invoice description priced anywhere from $30K to $300K. The gap is almost never the AI model. It’s whether the vendor looked at the data before they wrote the number.

Data preparation accounts for 60% to 80% of total effort on most production AI builds. If a proposal puts 10% of the budget on data prep, one of two things is true. Either nobody assessed your data, or the data work is buried inside another line item where you can’t see it. Both should worry you.

The four things that actually move the cost

When an AI quote swings wildly, the variance comes from four places. Three of them are not the model.

Data readiness. How much work happens before the model can do anything useful. Structured, consistent, well-labeled data is one project. Scattered, inconsistent, half-missing data is a fundamentally different one. This is the single biggest driver and the most underestimated.

Integration surface. Count the systems the AI has to connect to. One system is contained. Five systems, each with its own authentication, rate limits, and data schema, is a different scale of work. Each integration is not an API call. It’s auth, error handling, rate limit management, data transformation, and maintenance that never stops.

Model complexity. A reactive agent that takes one input and returns one output is the cheap end. A deliberative agent that reasons across multiple steps costs more. A multi-agent system that orchestrates several specialized agents is the most expensive category. If a vendor quotes a “simple AI” for a workflow that clearly needs multi-step reasoning, they have either misunderstood the task or are hoping you don’t notice.

Compliance. HIPAA for healthcare, SOC 2 and PCI for finance, GDPR for EU operations. Encryption, access logging, audit trails, and compliance documentation are real engineering hours. If your project carries these requirements and the proposal is silent on them, that scope hasn’t disappeared. It’s just waiting to arrive later as a change order.

Here’s how those costs actually break down on a production build:

Look at the distribution, not just the total. If 60% to 80% of a real project is data work and a quote shows 10%, the math is wrong somewhere. Ask any vendor to split their number into these categories and show you the percentages. A team that understands your project can do this in an afternoon. A team guessing at numbers will resist it.

How to read a quote before you sign it

You don’t need to be technical to pressure-test a proposal. You need to know what an honest one contains.

A complete AI proposal includes a data readiness assessment. It states the current condition of your data, what cleaning and structuring is required, who owns that work, and what happens if the data audit uncovers more than expected. If this section is missing, the rest of the numbers are guesses.

It includes a model selection rationale. Not just “we’ll use OpenAI,” but why that model fits your use case on cost, accuracy, and latency. A team that names a model without explaining the trade-off hasn’t thought hard about your requirements.

It includes an integration architecture that names every system. “Integrates with your CRM” is not a scope. “Connects to HubSpot via REST API, OAuth authentication, with these specific data transforms and this error handling” is a scope.

It defines edge case handling. What the AI does when it gets an input it doesn’t expect. What the fallback is when confidence is low. What triggers a human handoff. Production systems define this before the build, not after the first user complaint.

It includes monitoring. How you’ll know the model is working correctly once it’s live. Logging, alerting, accuracy dashboards. This is almost always absent from the cheap quote, because the cheap quote is priced as a demo, not a production system.

And it includes an ongoing maintenance estimate. Models drift. Integrations break. Edge cases accumulate. If the proposal says nothing about the cost of keeping the system alive, budget $1,500 to $3,000 a month and ask the vendor why they left it out. Before you compare upfront numbers, it’s worth understanding the full three-year cost of running an AI agent, because the cheapest build quote often becomes the most expensive project once monitoring and maintenance are counted.

Five red flags worth taking seriously

Some signals tell you, before any technical review, that a quote is built on assumptions rather than analysis.

No data preparation line item. Ask directly what’s included for data cleaning, formatting, and staging. If the answer is “we assume your data is clean,” the scope will grow the moment they see production data.

Vague deliverables. “An AI agent for your use case” with no specifics. Ask for week-by-week milestones. A real scope breaks into milestones. A vague one keeps room to underdeliver.

Production-ready in under four weeks. A four-week timeline for a complete AI system almost certainly skips data prep and production infrastructure. A fast demo and a production system are not the same thing. Ask what specifically is not included.

Open-ended time and materials with no cap. T&M is fine for genuine exploration. For a defined production build it creates cost exposure that compounds the moment data problems appear. And they always appear. Ask for fixed scope at fixed price, or a hard not-to-exceed number.

Technology choices with no justification. A named LLM, vector database, or cloud provider with no reason attached. Sometimes that reflects what the team knows best rather than what your project needs. Ask why this technology and not another.

What honest AI pricing actually looks like

Here’s a real reference. A single-workflow AI agent, say invoice processing for a mid-sized company with one ERP and one document source, costs $60K to $100K and takes 8 to 12 weeks. That includes the data readiness assessment, the model build, production infrastructure, monitoring, and 30 days of post-launch support.

A multi-workflow system with three or four integrations and messier data runs $100K to $180K over 12 to 16 weeks. A multi-agent system with custom orchestration, five or more integrations, and compliance requirements runs $180K to $400K over 16 to 20 weeks.

These are fixed-scope numbers, and the scope is set in week one from an actual data assessment, not from a brief. Notice the timelines. RAND’s research lines up here too. The failures often come from projects that should have been stopped early and weren’t, because nobody set honest checkpoints at three, six, or nine months. A scope defined against real data gives you those checkpoints.

The most reliable way to get an honest number is a short paid scoping engagement before you award the full build. Not a free discovery call. A two to three week paid engagement where the vendor assesses your data quality, documents your integrations, maps edge cases, and produces a fixed-price proposal with milestone-level scope. It costs $5K to $15K. It saves you from the $50K to $100K cost of discovering scope gaps mid-build. This is how a serious AI development partner scopes work, from the actual data and workflow rather than from a brief.

A vendor who resists that is telling you something. A vendor who welcomes it is building your proposal from analysis instead of hope.

So what should you do with the quote on your desk

Don’t sign the cheapest one because it’s clean. Clean usually means incomplete.

Take all three quotes and ask each vendor the same question: break this into data prep, integration, model, and compliance, and show me the percentages. The answers will sort the field fast. The vendor who can do it has looked at your problem. The vendor who can’t is selling you a demo and calling it a system.

The model was never the hard part. The data was. The quote that admits that is the one worth trusting.

Originally posted at RaftLabs