Structure Beats Data at the Boundary

How to combine machine learning with structured knowledge so your model extrapolates instead of flatlines

This is Part 3 of a 6-part series on simulation-grade AI for systems performance. Part 1 showed the extrapolation cliff. Part 2 introduced causal simulators. This post delivers the recipe that makes each mechanism in the graph extrapolate.

Recap: The Two Problems

Part 1 showed that pure ML models fail outside their training range: a gradient boosting model trained on servers 1–8 crashes to R² = -122% on servers 9–32. Part 2 introduced the causal graph, a DAG where each node’s mechanism can be independently learned and replaced [4].

But each mechanism in the graph still needs a model. And that model faces the same extrapolation cliff.

The insight is to factor each mechanism into two parts: one that handles what the model has seen (interpolation), and one that handles what it hasn’t (extrapolation via structured knowledge). Neither part works alone. Together, they cover the full range.

This pattern goes by different names depending on the domain. When the structured knowledge comes from physics (scaling laws, queuing theory, thermodynamics), it is called Physics-Informed Machine Learning (PIML) [3]. When it involves symbolic reasoning engines like logic programs or knowledge graphs, it falls under the broader neuro-symbolic AI research tradition [5]. In the general case, where any domain formula or constraint plays the structured role, it is sometimes called knowledge-informed ML. In this post, the structured knowledge comes from physics, so we are implementing the PIML variant. But the recipe is not limited to physics: wherever you have domain knowledge that can be expressed as a formula or constraint, the same factorization applies.

The Hybrid Factorization

The core idea is simple:

prediction = ML_model(data_parents) x structured_knowledge(formula_parents)

Data parents are variables that varied in the training data: configuration parameters, workload characteristics, concurrency levels. The ML model sees these during training and learns their effects through standard supervised learning. This is interpolation, and ML is good at it.

Formula parents are variables that were constant in training but change during deployment: hardware specs, node counts, data volumes. The ML model has never seen these vary, so it can’t learn their effects. Instead, we use structured knowledge from the domain: scaling laws, algebraic constraints, logical relationships, or any formula that encodes how the system must behave by construction.

The multiplication is the key. The ML model learns the base prediction as if hardware were fixed. The structured component adjusts that prediction for hardware changes. Neither part is asked to do what it can’t:

• The ML model never sees hardware features, so it can’t overfit to them
• The structured component doesn’t try to learn complex workload interactions; it only handles the constrained extrapolation

The structured component can take many forms depending on the domain:

• Physics laws: for hardware scaling relationships
• Algebraic identities: for derived quantities
• Logical constraints: for feasibility enforcement
• Domain formulas: from engineering, economics, or biology

In our database performance case, physics laws are the right choice. But the pattern (ML for complexity, symbolic knowledge for structure) is domain-agnostic.

Experiment 1: The Hybrid in Action

Let’s return to the synthetic experiment from Part 1 and reveal the solution.

The Setup (from Part 1)

Data generated from Amdahl’s Law [1]: response_time = base_load / speedup(N), where speedup(N) = 1 / (s + (1-s)/N). Here N is the number of servers and s is the serial fraction, the proportion of the workload that cannot be parallelized regardless of how many servers are added. We set s = 0.05, meaning 5% of the work is always serial. base_load depends on concurrent users (cu) and cache hit rate (ch), with 10 * cu * (1 - ch) + 5 capturing the effective miss load. Training on servers 1–8, testing on 9–32:

import numpy as np

np.random.seed(42)
n_samples = 500
s_true = 0.05  # true serial fraction: what we will try to recover

def amdahl_spd(N, s):
    return 1.0 / (s + (1 - s) / N)

# Server counts: 1-8 for training, 9-32 for testing (unseen during training)
N_train = np.random.randint(1, 9,  n_samples).astype(float)
N_test  = np.random.randint(9, 33, n_samples).astype(float)

# Workload features: concurrent users and cache hit rate
cu_train = np.random.uniform(1, 20,   n_samples)
ch_train = np.random.uniform(0.5, 0.95, n_samples)
cu_test  = np.random.uniform(1, 20,   n_samples)
ch_test  = np.random.uniform(0.5, 0.95, n_samples)

# Response time = workload-driven base load / Amdahl speedup + noise
base_train = 10 * cu_train * (1 - ch_train) + 5
base_test  = 10 * cu_test  * (1 - ch_test)  + 5

y_train = base_train / amdahl_spd(N_train, s_true) + np.random.normal(0, 0.5, n_samples)
y_test  = base_test  / amdahl_spd(N_test,  s_true) + np.random.normal(0, 0.5, n_samples)

The PIML Model

If we assume response = base_load / speedup(N, s), we can rearrange to isolate the base load: base_load = response x speedup(N, s). For the right value of s, this "undone" target should depend only on workload features, not on server count.

The key insight is that for the correct value of s, multiplying the observed response time by speedup(N, s) should strip out the hardware scaling effect entirely, leaving a residual that depends only on workload (concurrent users and cache hit rate) and not on N at all. We exploit this by scanning 50 candidate values of s between 0.01 and 0.3. For each, we undo the scaling, fit a simple linear model on the workload features alone, and measure how well it explains the residual. The best s is the one that produces the cleanest workload signal, the highest R² from workload features only. The workload features are concurrent users (cu), cache hit rate (ch), and their interaction term cu * (1 - ch), which captures effective cache-miss load:

from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score

# Grid search over serial fraction s.
# For each candidate s, we undo the hardware scaling and ask: can workload
# features alone explain what's left? The correct s strips N out entirely,
# so workload features explain the residual perfectly. A wrong s leaves a
# hidden N-dependence that workload features can't capture, and R² stays low.
best_s, best_score = 0.05, -np.inf
for s in np.linspace(0.01, 0.3, 50):
    spd = amdahl_spd(N_train, s)
    base_targets = y_train * spd           # undo hardware scaling
    X_base = np.column_stack([cu_train, ch_train, cu_train * (1 - ch_train)])
    # LinearRegression is intentionally used here as a fast scoring function,
    # not the final model. Running GBT 50 times would be slow and could
    # overfit each candidate, making it harder to identify the true optimum.
    reg = LinearRegression().fit(X_base, base_targets)
    full_pred = reg.predict(X_base) / spd
    sc = r2_score(y_train, full_pred)
    if sc > best_score:
        best_score, best_s = sc, s
print(f"Learned s = {best_s:.4f} (true = 0.05)")
# Output: Learned s = 0.0514

The grid search recovers s = 0.0514, close to the true value of 0.05, learned entirely from data.

This is a deliberate two-stage design. The linear model inside the grid search is not the final predictor; it is a fast scoring function used to identify the s that best decouples hardware from workload. Running gradient boosting 50 times would be slow and could overfit to individual candidate values, making it harder to find the true optimum. Linear regression is intentionally weak here: stable, fast, and sufficient to tell whether a given s successfully strips the hardware effect from the signal.

Once s is known, the target y_train * spd_final is a clean, hardware-free workload signal. We then train a gradient boosting model on it, one expressive enough to capture nonlinear workload interactions that the linear model would miss, such as how cache hit rate modulates the effect of concurrency on base load. At prediction time, the two components chain together: the GBT predicts base load from workload features, and the formula scales that prediction to any N. Crucially, N is never passed to the ML model; it only enters through the formula:

spd_final = amdahl_spd(N_train, best_s)
base_model = HistGradientBoostingRegressor(max_depth=4, max_iter=100)

base_model.fit(
    np.column_stack([cu_train, ch_train, cu_train * (1 - ch_train)]),
    y_train * spd_final   # target is the "undone" base load
)

def hybrid_predict(X):
    N, cu, ch = X[:, 0], X[:, 1], X[:, 2]
    base = base_model.predict(np.column_stack([cu, ch, cu * (1 - ch)]))
    return base / amdahl_spd(N, best_s)

By keeping N out of the ML model entirely, we prevent it from learning a spurious relationship with server count that would break down outside the training range. The ML model is responsible only for what it has seen; the formula handles everything it hasn't.

Results

The hybrid nearly matches the oracle, without being given the true parameters. It learned the serial fraction from data and used it to extrapolate. The pure ML model, despite being a powerful gradient boosting regressor, flatlines the moment it leaves the training range. This is a structural property of tree-based models: decision tree leaves partition the feature space into regions, so predictions outside the training range are held constant at the nearest leaf value; they cannot extrapolate a trend they haven’t seen.

The difference isn’t model complexity. It’s structure.

Experiment 2: What If the Structure Is Wrong?

A fair criticism: “This only works because you used the right formula.”

What happens when the assumed structure is wrong?

We regenerate the data from the Universal Scalability Law (USL) [2], which includes a coherency penalty that Amdahl’s Law ignores:

USL throughput(N) = N / (1 + sigma*(N-1) + kappa*N*(N-1))

At high N, USL throughput actually drops (coherency overhead dominates), while Amdahl's merely plateaus. This creates a genuine structural mismatch: we assume Amdahl (monotonic improvement), but the data comes from USL (improvement then degradation).

The data-generating function below replaces the Amdahl DGP from Experiment 1. The base term encodes the same workload structure as before; only the scaling law changes. sigma controls contention overhead (linear in N) and kappa controls coherency overhead (quadratic in N), which is what causes throughput to peak and then fall:

def usl_rt(N, cu, ch, sigma=0.03, kappa=0.002):
    base = 10 * cu * (1 - ch) + 5
    throughput = N / (1 + sigma * (N - 1) + kappa * N * (N - 1))
    return base / throughput

We train on servers 1–8, where USL and Amdahl look nearly identical (the coherency term is still small), and test on 9–32, where the quadratic penalty causes USL throughput to peak and decline while Amdahl continues to improve. The hybrid is trained using the same grid search procedure as Experiment 1, still assuming Amdahl, the wrong functional form.

Even with the wrong structure, the hybrid outperforms pure ML by 29 percentage points. The approximate structural assumption (that performance follows some constrained scaling relationship) is enough to guide the model in the right direction.

This matters in practice. You rarely know the exact relationship in a real system. But even an approximate structural assumption goes a long way. This is what makes knowledge-informed approaches robust: you don’t need a perfect symbolic theory, just a useful one.

Applying the Recipe to the Real Graph

The experiments above used synthetic data with a single known law. In the real causal graph, the same factorisation applies across multiple edges, but the right structured component differs per edge depending on what the architecture tells us about that relationship.

The graph’s topology is itself a form of encoded knowledge. Node Count has no direct edge to any performance metric; its effect is fully mediated through intermediate nodes:

Node Count → Total Parallelism → Data Per Processor → Performance Metrics

This is not an accident. In a shared-nothing database architecture, adding nodes doesn’t speed up individual operations; it redistributes data. Each node handles less data, so each node does less work. Under ideal conditions, performance approaches linear scaling with node count. Encoding this as mediation rather than a direct edge forces the model to respect this structure. A pure ML model with a direct edge from Node Count to metrics could learn any relationship the data supports, including spurious ones. The graph topology rules that out by construction.

Once the topology is fixed, each edge needs a structured component matched to what we know about that mechanism. For CPU-bound metrics under parallelism, Amdahl’s Law may be a natural choice: it models speedup as bounded by a serial fraction, which may fit systems where adding processors helps but bottlenecks remain. For I/O metrics driven by data redistribution, a power law may be appropriate: I/O volume tends to scale with how much data each processor handles, though the right exponent depends on the workload. For memory-related metrics, a sublinear variant may be a better fit; caching and buffer reuse often mean memory doesn’t grow in direct proportion to data volume per processor. These are starting points informed by domain knowledge, not prescriptions. The right structured component for any edge should be validated against observed behavior.

In a real deployment scenario, the grid search approach faces a practical constraint: it requires the scaling variable to vary meaningfully in the training data. If NodeCount were constant across all observations, as is common in production logs from a fixed-hardware system, there would be no variation for the search to exploit, and s could not be recovered this way. In that case, the structured component's parameters would need to come from calibrated engineering defaults derived from domain knowledge of the architecture rather than being fitted on the fly. The experiment above demonstrates the technique under conditions where it works. In practice, knowing when those conditions hold is part of applying it correctly.

Key Takeaways

The recipe is simple: identify what you know structurally about a relationship, encode it explicitly, and let the ML model handle what it can actually learn from data. Neither perspective is sufficient on its own. ML without structure overfits to the observed range. Structure without ML can’t capture complex interactions the formula doesn’t account for. Together, they cover the full range.

Two results from the experiments are worth holding onto. First, the hybrid nearly matches an oracle that knows the true parameters, not because it was given them, but because the right structure lets the data reveal them. Second, even the wrong structure outperforms no structure. You don’t need perfect physics. You need a useful constraint.

The graph takes this further: structure lives not just in the node mechanisms but in the topology itself. Encoding the shared-nothing redistribution property as mediation rather than a direct edge is an architectural decision that no amount of training data could enforce on its own.

What’s Next

We have a model that extrapolates. But so far, we’ve only asked it to predict: “what will Y be given X?”

The more powerful question is: “What happens if I do X?” Not “what was Y when X was high in the past?” but “what will Y be if I force X to a new value, regardless of what usually accompanies X?”

The difference between these questions is the difference between observation and intervention, and getting it wrong can lead to exactly the wrong decision. That’s the subject of the next post.

References

[1] Amdahl, G.M. (1967). Validity of the Single Processor Approach. AFIPS Proceedings. https://dl.acm.org/doi/10.1145/1465482.1465560

[2] Gunther, N.J. (2008). A General Theory of Computational Scalability Based on Rational Functions. arXiv:0808.1431. https://arxiv.org/abs/0808.1431

[3] Meng, Chuizheng, Sam Griesemer, Defu Cao, Sungyong Seo, and Yan Liu. “When physics meets machine learning: A survey of physics-informed machine learning.” Machine Learning for Computational Science and Engineering 1, no. 1 (2025): 20. https://link.springer.com/article/10.1007/s44379-025-00016-0

[4] Blöbaum, Patrick, Peter Götz, Kailash Budhathoki, Atalanti A. Mastakouri, and Dominik Janzing. “DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models.” Journal of Machine Learning Research 25, no. 147 (2024): 1–7. https://jmlr.org/papers/v25/22-1258.html

[5] Bhuyan, Bikram Pratim, Amar Ramdane-Cherif, Ravi Tomar, and T. P. Singh. “Neuro-symbolic artificial intelligence: a survey.” Neural Computing and Applications 36, no. 21 (2024): 12809–12844. https://link.springer.com/article/10.1007/s00521-024-09960-z

This post is part of the Teradata Labs series on simulation analytics for enterprise systems. We publish technical perspectives on causal modeling, physics-informed ML, and structural simulation for reliable performance prediction under real-world conditions.

Next in the series: Part 4: Do, Don’t Just Predict: Interventions and Counterfactuals for Systems Engineers