ML Is a Photograph. Systems Need a Blueprint.

Why every other engineering discipline uses simulators and why systems performance should too

This is Part 2 of a series on simulation-grade AI for systems performance. Part 1 showed why ML models fail at extrapolation. This post introduces the paradigm that fixes it.

The simulator mindset

In Part 1, we saw how ML prediction models fail on workload performance tasks: they score well on held-out data, then collapse the moment conditions change. The model is fine until it’s asked about a configuration it’s never seen.

The question now is: what kind of system doesn’t fail when inputs change?

Every mature engineering discipline already has an answer. Simulation.

When reality is expensive, dangerous, or simply not testable, engineers don’t rely on historical data alone. They build models of how the system works.

• You don’t run nuclear tests to study failure modes. The consequences are catastrophic, and real failures are too rare to form a useful dataset.
• You don’t crash thousands of cars to design a safer bumper. Each crash destroys a prototype, so you’ll never collect enough samples to cover the design space.
• You don’t deliberately start an epidemic to study transmission. The ethics forbid it, and every new pathogen behaves differently from the last, so historical outbreaks don’t generalize.

You simulate.

Scenarios where traditional prediction fails to scale, yet simulation-based modeling provides insights. Image generated using Gemini.

But there’s a deeper reason simulation matters, especially for systems performance. A good simulator doesn’t just reproduce what you’ve already seen. It can reason about what it has never observed.

Boeing doesn’t predict whether a wing will fail solely by training on historical crash data. They combine that data with simulated airflow from computational fluid dynamics. Change the wing design, and the simulation still works, because it captures the underlying physics rather than the patterns of past failures.

Formula 1 teams don’t tune their cars purely by collecting more lap data. They combine race telemetry with simulations of aerodynamics, tire degradation, and race strategy. When a new track condition appears, the simulation layer can still reason about it.

Crash-test engineers don’t need a database of every possible collision. They simulate force, deformation, and energy transfer, and get useful answers for scenarios they’ve never observed.

These fields solved the extrapolation problem decades ago, not by collecting more data, but by changing the modeling paradigm.

So why are we still doing pattern matching for systems performance?

Prediction vs. simulation

Part of the problem is that we’ve been treating prediction and simulation as if they were the same thing. Let’s pull them apart.

A prediction model learns a mapping from inputs to outputs. A simulator models how outputs are generated.

Concretely: given features X (server count, cache size, workload), a prediction model learns f(X) → Y from data, where Y is an outcome like CPU time or I/O counts. A simulator doesn't learn that mapping directly. It models how X influences intermediate variables, which in turn influence Y, through a chain of cause-and-effect relationships.

Each link in that chain is what we call a mechanism: the rule that determines a variable’s value from its causes. The mechanism can be a formula, a learned model, or a hybrid; the SCM section below makes this precise.

That difference starts to matter the moment inputs change:

How each approach handles different situations. On seen data they tie. On everything else, the gap widens.

A prediction model is a photograph. It captures the relationship at the moment the data was collected. A simulator is a blueprint. It describes how the parts connect and interact, and it’s still valid when the parts change.

The key difference comes down to the question each one answers.

A prediction model answers: “What usually happens when X occurs?”

A simulator answers: “What would happen if X changed?”

That shift, from correlation to “mechanisms” within simulation, is what enables extrapolation.

Structural causal models: a practical framework

To make this concrete, we need a way to represent “mechanisms” in a system. The most useful framework comes from causal inference: Structural Causal Models (SCMs), introduced by Judea Pearl [1]. Python libraries like DoWhy-GCM [2] implement this directly.

An SCM gives you three things:

1. A graph of cause-and-effect relationships
2. A mechanism for each relationship
3. A way to reason about interventions and counterfactuals

Instead of treating the system as a black box, you explicitly model how variables influence each other.

Here’s what each of those three pieces actually is.

A directed acyclic graph. Nodes are variables. Edges represent causal mechanisms, with “A causes B” drawn A → B. “Acyclic” means no variable can cause itself through a chain of effects.

A mechanism for each edge. Each node’s value is a function of its parent nodes plus noise: X_i = f_i(parents(X_i), noise_i). The function can be a formula, an ML model, or a hybrid of both.

Independent noise terms. Each node has its own source of randomness, independent of the others. That independence is what makes the model structural rather than just statistical.

A simple example. Consider a web server with three variables:

Concurrent Users → CPU Utilization → Response Time

The DAG says: concurrent users cause CPU utilization (more users, more processing), and CPU utilization causes response time (higher CPU, slower responses). Each arrow carries a mechanism:

• Users → CPU: cpu = f1(users) + noise1 (maybe a learned ML model)
• CPU → Response Time: rt = f2(cpu) + noise2 (maybe a formula from queuing theory)

It looks simple. But the structure gives you something useful: modularity.

What the graph buys you

Interventions

A classic example before we get technical. An ML model sifts through beach data and notices that ice cream sales and shark attacks rise together every summer. Presto (remember him from Part 1) reads the pattern and calls it: ice cream attracts sharks. Ban the ice cream, save the swimmers.

Sim looks at the same data and asks what else is going on. It’s heat. People buy ice cream when it’s hot. They also swim when it’s hot, which puts more swimmers in the water with sharks. Ban the ice cream, keep the heat, and the shark attacks don’t budge.

Storyboard generated by Gemini

That gap, between “these move together” and “this causes that,” is where prediction models quietly mislead, and where an SCM earns its keep.

For a less whimsical example, a prediction model for your database can tell you: “when cache hit rate was 80% in the past, CPU was high.” Fair enough. But cache was 80% during peak hours, when concurrency was also high. The model learned an association, not a causal effect.

An SCM lets you perform an intervention: force cache hit rate to 80% while holding everything else at its natural state. In graph terms, you cut the incoming edges to the cache node and set its value by hand. That removes the confounding influence of peak hours and shows the actual causal effect of cache on CPU.

Pearl writes this as do(cache = 80%), which is a different operation from observe(cache = 80%). Part 4 of the series goes deep on this.

The difference between observing and intervening is what most ML systems miss.

Counterfactuals

“Last Tuesday at 3pm, response time spiked to 500ms. Would the spike have been avoided if we’d doubled the cache at 2pm?”

A prediction model can’t answer this. It can predict future outcomes given inputs, but it can’t reason about alternative pasts. An SCM can, because it models the mechanisms that generated Tuesday’s spike. You keep the same noise terms (the same workload, the same random fluctuations) and change only the cache setting. What comes back is a counterfactual: a statement about what would have happened under different conditions.

Root cause analysis

When a metric spikes, a prediction model gives you feature importance: “server count and cache rate were the most important features.” That tells you what correlated with the spike in the training data. It doesn’t tell you what caused it.

In an SCM, you trace the arrows backward. If response time spiked, check the immediate parents: was it CPU? I/O? Spool? Whichever one spiked, trace further: configuration change, workload shift, hardware issue? The graph gives you a causal explanation, not a statistical one.

Modularity

This is the part people sleep on. Each mechanism in the graph is independently learnable and replaceable.

Suppose you upgrade from spinning disks to SSDs. A prediction model has to retrain from scratch, because the entire model has implicitly encoded disk speed into every learned pattern. In an SCM, you replace one mechanism: the edge from storage type to I/O latency. Everything else (the CPU model, the memory model, the spool model) stays untouched.

This is how engineering simulators work. Boeing doesn’t rebuild their entire CFD model when they change the wing material. They update the material-properties module.

That’s the quiet advantage of simulation. You don’t retrain the system. You update the part that changed.

From theory to architecture

For a real system like a database, this structure naturally organizes into layers. Recent VLDB work on applying causal inference to database query logs [3] makes a related case.

Layer 1, roots. Things you control or that define the environment. Hardware (node count, memory per node, processors per node), data volume, configuration parameters. These are the knobs you turn.

Layer 2, intermediates. Variables you don’t directly control, but that follow from the roots. Some of these are deterministic. 8 nodes with 18 processors each gives 144-way parallelism; 1TB spread across 144 processors gives about 7GB per processor. Others are better captured by a learned model when the relationship exists but isn’t clean enough to write down as a formula. Either way, the node sits in the middle of the graph rather than at its edge, and the choice between rule-based and ML is per-edge, not per-layer.

Layer 3, performance metrics. The things you measure and care about: CPU time, memory usage, I/O volume, spool consumption. These depend on the configuration parameters from Layer 1 and on the intermediates from Layer 2, through mechanisms that combine ML and domain knowledge.

One thing to notice: hardware effects are mediated. Adding more nodes doesn’t directly reduce CPU time. It increases total parallelism, which reduces the data each processor handles, which reduces CPU time per query. The DAG makes this explicit:

Node Count → Total Parallelism → Data Per Processor → CPU Time

There is no direct edge from Node Count to CPU Time. The effect is fully mediated through the intermediates. That detail will matter for extrapolation, which is the subject of the next post.

What a simulator is not

A few things to be clear about.

A simulator in this sense is not a digital twin. A digital twin is a live, synchronized replica of a specific system. Our simulator models the underlying mechanisms instead, and it can represent any configuration, not just the current one.

The simulator also doesn’t replace monitoring. It predicts what would happen under a given configuration; monitoring tells you what is happening right now. Different jobs.

And no, the simulator is not perfect. The DAG is a model, and all models are wrong. If a causal relationship is missing from the graph, the simulator can’t reason about it. The upside is that when the graph is approximately right, the simulator degrades gracefully, where a prediction model can fail catastrophically the moment it steps outside its training range.

The catch

Structure alone isn’t enough.

Each edge in the graph still needs a model. And those models face the same limitation from Part 1: if they’re trained only on observed data, they won’t extrapolate.

The DAG gives you the wiring. But the behavior of each component still matters.

If the mechanism itself doesn’t generalize, the simulator won’t either.

That leads to the next post in the series. We need models that combine data with domain knowledge, so each mechanism can extrapolate, not just interpolate.

References

[1] Pearl, Judea. *Causality*. Cambridge university press, 2009.

[2] Blöbaum, Patrick, Peter Götz, Kailash Budhathoki, Atalanti A. Mastakouri, and Dominik Janzing. “DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models.” Journal of Machine Learning Research 25, no. 147 (2024): 1–7.

[3] Markakis, Markos, Brit Youngmann, Trinity Gao, Ziyu Zhang, Rana Shahout, Peter Baile Chen, Chunwei Liu, Ibrahim Sabek, and Michael Cafarella. “From logs to causal inference: diagnosing large systems.” Proceedings of the VLDB Endowment 18, no. 2 (2024): 158–172.

This post is part of the Teradata Labs series on simulation analytics for enterprise systems. We publish technical perspectives on causal modeling, physics-informed ML, and structural simulation for reliable performance prediction under real-world conditions. Next in the series: Part 3: Structure Beats Data at the Boundary