Your ML Model Is 95% Accurate: On Data It’s Already Seen

The extrapolation problem in ML-based system performance prediction

This is Part 1 of a series on simulation-grade AI for systems performance. This post shows why ML models fail at extrapolation. The rest of the series builds the fix.

The Problem

Monday morning. Your capacity plan is due. The cluster is jumping from 8 nodes to 16, and leadership wants to know: will queries still meet SLA?

This is workload modeling: predicting how a system will behave before you change anything. You trained an ML model on six months of query logs. It scored 95% on held-out data. You run the prediction. The numbers look fine. You ship it.

Capacity planners, DBAs, SREs, and performance engineers have been answering questions like this for decades. Will the query meet SLA on new hardware? Can we handle 5× more users? What breaks if data doubles? The questions haven’t changed. The tools have. Analytical formulas gave way to ML models trained on historical query logs, promising accuracy the old formulas couldn’t touch.

Here’s the problem. That 95% was measured on data the model had already seen. Deploy it anywhere something has changed: new hardware, an untried configuration, an unfamiliar workload. The prediction isn’t just off. It can be worse than guessing the mean.

To understand why, meet Presto and Sim.

Storyboard generated by Gemini

Presto the Predictor has studied years of desert weather data and learned every pattern in it. He can tell you the temperature for any day in June with remarkable accuracy. But when it rains for the first time, he’s lost. Rain lies outside his training distribution. He defaults to what he knows: “The ground stays dry.”

Sim the Scientist has never seen rain either, but she understands the underlying mechanisms: gravity, fluid dynamics, evaporation. She doesn’t need historical examples of rain to reason about what happens when water falls from the sky.

Now replace “desert vs. rain” with “8-node clusters vs. 16-node clusters.”

Your company upgrades its database hardware. The ML model has never seen 16 nodes. It still returns a confident prediction, but it’s based on patterns that no longer apply. Meanwhile, a model that captures how performance scales with parallelism continues to behave sensibly.

This is the extrapolation problem. And in real systems, it’s often the most dangerous failure mode: the model fails precisely when you need it most.

Why ML Models Struggle to Extrapolate

Most standard machine learning models trained on observational data behave as interpolators. They learn patterns within the range of data they’ve seen.

When inputs move outside that range, predictions are no longer anchored by data. Instead, they are driven by the model’s inductive biases.

• Tree-based models (XGBoost, LightGBM, random forests) produce piecewise constant outputs. In practice, their predictions are often bounded by values observed during training [1]. Ask a model trained on response times between 50 ms and 200 ms what happens at 64 servers, and it will typically return something in that range, even if the true answer is much lower.
• Neural networks can produce outputs beyond the training range, but on out-of-distribution inputs they often exhibit unstable or overly simplistic behavior [2], rather than meaningful extrapolation.

At a deeper level, the issue is structural: the model has learned correlations within a region, but not the mechanism that governs behavior outside it.

There are important exceptions. A linear model can extrapolate when the true relationship is linear. More broadly, models that incorporate structure, constraints, or domain knowledge can generalize better beyond observed data.

But in complex systems with nonlinear interactions and hidden variables, models trained purely on historical observations typically degrade when pushed outside their training distribution [[3]](http://Dakhmouche, R. & Gorji, H. (2025). Why Cannot Neural Networks Master Extrapolation?).

More data within the same regime usually doesn’t fix this. Increasing model complexity alone doesn’t fix it either. The core limitation is often lack of structural understanding, not just lack of data.

Experiment: The Extrapolation Cliff

Setup

To make this concrete, we generate synthetic server performance data using **Amdahl’s Law**, a foundational result in parallel computing.

In simple terms: adding more servers improves performance, but not indefinitely. Some fraction of the workload is inherently serial (e.g., coordination, locking), which limits how much speedup is possible.

If 5% of the work is serial, even infinite servers can only yield about a 20× speedup.

The formulation is:

response_time = base_load / speedup(N)
speedup(N) = 1 / (s + (1 - s) / N)

where:

• N = number of servers
• s = 0.05 = serial fraction
• base_load depends on workload characteristics

We train a model on servers 1–8, and evaluate on:

• In-distribution: servers 1–8
• Out-of-distribution: servers 9–32

We compare:

• Pure ML: Gradient boosting trained on all features
• Analytical Oracle: The true generating function (Amdahl’s Law with known parameters)

In practice, we don’t know the true function. This oracle serves as an upper bound on achievable performance.

import numpy as np
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.metrics import r2_score

np.random.seed(42)
TRUE_S, TRUE_ALPHA, TRUE_BETA = 0.05, 10.0, 5.0

def true_rt(N, cu, ch):
    """Ground truth: response_time = base_load / speedup.
    N  = server count
    cu = concurrent users
    ch = cache hit rate (0-1)
    """
    base = TRUE_ALPHA * cu * (1 - ch) + TRUE_BETA
    speedup = 1.0 / (TRUE_S + (1 - TRUE_S) / N)
    return base / speedup

def gen(n, srange):
    rows = []
    for _ in range(n):
        N  = np.random.choice(list(srange))                          # server count
        cu = np.random.uniform(10, 200)                              # concurrent users
        ch = np.random.uniform(0.3, 0.95)                            # cache hit rate
        rt = max(true_rt(N, cu, ch) + np.random.normal(0, 2), 0.1)   # response time (ms)
        rows.append([N, cu, ch, rt])
    return np.array(rows)

train = gen(500, range(1, 9))       # servers 1-8
test_id = gen(200, range(1, 9))     # in-distribution
test_ood = gen(300, range(9, 33))   # out-of-distribution

X_tr, y_tr = train[:, :3], train[:, 3]
X_id, y_id = test_id[:, :3], test_id[:, 3]
X_ood, y_ood = test_ood[:, :3], test_ood[:, 3]

# Pure ML: gradient boosting on all features
ml = HistGradientBoostingRegressor(max_depth=6, max_iter=200, random_state=42)
ml.fit(X_tr, y_tr)

Results

Look at that OOD R². Not 80%. Not 50%. Negative 122%. The model is performing worse than predicting the mean for every input.

In-distribution, everything looks fine: 95.8% would pass most model reviews. The failure is invisible until deployment, when the inputs shift to a regime the model has never seen.

Figure 1. Scaling curve: the pure ML model flatlines at its last training value (server 8). The analytical oracle continues to follow the true curve into unseen territory. The gap between them is the extrapolation cliff.

Why This Matters in Practice

This isn’t an academic exercise. Every production system encounters the extrapolation cliff:

• Hardware upgrades. You trained on 8 nodes. The cluster doubles to 16. Every prediction is now out-of-distribution.
• Configuration changes. Someone adjusts a parameter combination that’s never been tried. The model has no basis for its prediction.
• Workload shifts. Black Friday traffic is 5x normal. The model extrapolates from patterns learned at 1x.
• Capacity planning. The entire purpose of capacity planning is to predict performance at scales you haven’t reached yet, which is, by definition, extrapolation.

The RAND Corporation’s [2024 analysis of AI project failures](http://RAND Corporation (2024). The Root Causes of Failure for AI Projects. Report RRA2680-1.) found that distribution shift and out-of-distribution deployment are among the most common root causes. The model works in development, passes testing, and fails in production, because production is where the inputs change.

The insidious part is that the model doesn’t warn you. Gradient boosting doesn’t return “I don’t know.” It returns a confident number that happens to be wrong.

So What Do We Do?

More data helps if you can collect it across the full range of conditions you’ll encounter. But if you could do that, you wouldn’t need a prediction model. You’d already have the answers.

More complex models don’t help. A deeper neural network or a larger ensemble faces the same structural limitation: the model has never seen the region it’s being asked about.

The answer isn’t a better model. It’s a different paradigm, one borrowed from fields that solved the extrapolation problem decades ago.

The key shift is not abandoning machine learning, but changing what the model is learning.

We need models that combine learned patterns with structure and mechanisms drawn from domain knowledge.

Boeing doesn’t predict wing failure from crash data alone. They simulate airflow with computational fluid dynamics alongside it. Formula 1 teams don’t tune their cars from lap data alone. They simulate aerodynamics, tire degradation, and race strategy alongside it. These systems work outside their training data because they combine mechanism-based modeling with patterns, not because they abandon data altogether.

In the next post, we’ll explore what changes when you shift from prediction to simulation, and how a framework from causal inference gives us the tools to build simulators for systems performance.

References

[1] Malistov, Alexey, and Arseniy Trushin. “Gradient boosted trees with extrapolation.” In 2019 18th IEEE international conference on machine learning and applications (ICMLA), pp. 783–789. IEEE, 2019.

[2] Kang, Katie, Amrith Setlur, Claire Tomlin, and Sergey Levine. “Deep Neural Networks Tend To Extrapolate Predictably.” In The Twelfth International Conference on Learning Representations.

[3] Dakhmouche, Ramzi, and Hossein Gorji. “Why Cannot Neural Networks Master Extrapolation? Insights from Physical Laws.” arXiv preprint arXiv:2510.04102 (2025). 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop: Machine Learning and the Physical Sciences.

This post is part of the Teradata Labs series on simulation analytics for enterprise systems. We publish technical perspectives on causal modeling, physics-informed ML, and structural simulation for reliable performance prediction under real-world conditions. Next in the series: Part 2: ML Is a Photograph. Systems Need a Blueprint.