Zero-Knob Bayesian MMM: I Taught a Neural Network to Configure Marketing Models — and Built the Gate That Catches It When It’s Wrong

What happens when a machine learns the expertise that made marketing measurement a consulting business — with working code, real calibration numbers, and the trust architecture that makes it safe. From the Papilon research program at Vector1 Research.

Every serious marketing mix model ships with a hidden dependency: a human expert.

Whether it’s Google’s Meridian, Meta’s Robyn, or a hand-rolled PyMC model, someone has to set the knobs — how long an ad’s effect lingers (adstock priors), where diminishing returns kick in (saturation parameterizations), how much small markets should borrow strength from big ones (hierarchical pooling), how often to refit. Those choices are where the expertise lives. They’re also where the errors hide, and why every MMM deployment quietly turns into a consulting engagement. The model is open-source; the knowledge of what the knobs mean is not.

For the past stretch of the Papilon research program at Vector1, I’ve been working on a blunt question: can a machine learn the configuration itself — and if it can, how would we ever trust it?

This article reports the answer from three working prototypes: yes, it can — and the trust question turns out to have a beautiful engineering solution. Everything below is runnable, dependency-free code, public in the Papilon repo.

The full paper, with the formal treatment and all three experiments, is published as a preprint: doi.org/10.5281/zenodo.21779665. What follows is the readable version.

The idea in plain terms: a doctor who has seen a million patients

The technical name for the approach is amortized inference, but the intuition is simple. A veteran doctor doesn’t re-derive medicine from first principles for each patient — she has seen so many cases that she recognizes the pattern on sight, and orders the confirming test. The expertise was paid for up front, across thousands of prior cases, so each new diagnosis is fast.

Now apply that to marketing models. This is exactly what PyCausalSim, my open-source framework for causal discovery through simulation, makes possible: it can generate unlimited synthetic marketing environments where the ground truth is known — every adstock decay rate, every saturation curve, every feedback loop, chosen by us. Train a neural network on tens of thousands of these simulated worlds, and it learns to look at a raw marketing panel and recognize the structure: this looks like slow carryover, early saturation, a strong search channel. At deployment, configuration happens in a forward pass — seconds, not weeks. The machine learned in simulation what consultants learned over careers.

The academic lineage is real and recent — this is simulation-based inference, the same family as TabPFN, which stunned the tabular-ML world by pre-training a network on millions of synthetic datasets. Nobody had built the marketing-measurement instance. That’s the whitespace this program occupies.

The obvious objection — and the design move that answers it

The doctor analogy contains its own warning: a doctor who trained only on textbook cases will confidently misdiagnose the patient whose disease isn’t in the textbook. Our network is only as good as PyCausalSim’s simulated worlds. If reality falls outside them, the network won’t fail loudly — it will project reality onto the nearest simulation it knows and be confidently wrong. In a system whose output moves media budgets, that’s disqualifying.

So the entire architecture rests on one principle:

Nothing the meta-learner outputs is a verdict. Everything is a proposal that the mathematics audits.

One scalar routes every dataset: exactness for free when the network is right, loud failure when it isn’t.

Here’s how the audit works, and it’s the part I’m proudest of because it uses fifty-year-old statistics to discipline brand-new machine learning. The network’s answer — a probability distribution over the model’s parameters — is treated as an importance-sampling proposal and checked against the true model likelihood, the exact Bayesian math we’d normally grind through with slow MCMC sampling. The check produces one number: the effective sample size (ESS) of the importance weights. Intuitively, ESS asks: out of 4,000 answers the network suggested, how many does the actual mathematics agree are plausible?

theta = q.sample(n=4000) # amortized draws — free

log_w = model.logp(theta, D) — q.logp(theta) # exact reweighting

ess = effective_sample_size(log_w) / n

if ess > 0.10: accept(reweight(theta, log_w)) # asymptotically EXACT

elif ess > 0.01: refine(mcmc_seeded_at(theta)) # cheap correction

else: escalate(full_mcmc()); log_gap() # simulator gap episode

Three outcomes. If the network’s proposal agrees with the math (high ESS), the reweighted answer is asymptotically exact — you get textbook Bayesian inference at neural-network speed, for free. If it’s close but imperfect, a short MCMC run seeded at the network’s best guess corrects it cheaply. And if the network is badly wrong, the ESS says so quantitatively, before any number reaches a human — the system falls back to full classical inference, and the failure is logged as a gap in the simulator’s training worlds, to be fixed in the next round.

The failure mode of a bad network is that inference gets slower. It cannot get wrong. That is the trust contract — and unlike most claims about AI safety layers, this one is demonstrated below rather than asserted.

Result 1: the gate works — quality only ever costs compute

Prototype 1 stress-tests the gate against a miniature MMM (adstock + Hill saturation, 120 weeks of data) where I know the true answer, feeding it proposals of deliberately varying quality:

Measured ESS ratios across four proposal regimes. Proposal quality affects cost, never correctness.

Read the last column. Every regime recovered the truth. A great proposal earned exact inference with zero sampling; a mediocre one paid for a short correction; a hopeless one triggered the full classical fallback. Four regimes, four costs, one answer.

The prototyping also surfaced a finding I hadn’t planned for. The toy model’s posterior carries a 0.93 correlation between the saturation parameter and the channel coefficient. In plain terms: the data genuinely cannot tell apart “a steep response curve with a big multiplier” from “a shallow curve with a small one” — a classic identifiability trade-off every MMM practitioner eventually gets burned by. That correlation has a practical consequence for the architecture: a proposal that ignores it (a “diagonal” or mean-field posterior, the lazy default) scores ESS ≈ 0.21 even when placed at exactly the right location, while the full-covariance version scores 0.70. Mean-field heads are empirically disqualified; the meta-learner must model how parameters trade off against each other. The design’s insistence on full-covariance and normalizing-flow heads went from taste to measurement.

Result 2: even the network’s model choices get overruled by evidence

Configuration isn’t only continuous dials — the system must also choose between model families. Does this channel’s effect peak immediately and decay (geometric adstock, typical of search), or peak days later (delayed adstock, typical of CTV and upper-funnel media)? Get the family wrong and every downstream number inherits the error.

Prototype 2 shows the family choice inherits the same safety property, through a reuse I find genuinely elegant: the same importance weights that audit the continuous parameters also yield an unbiased estimate of each family’s marginal likelihood — the total evidence the data provides for that family (Ẑ = the average of p/q). The corrected model choice is then just P(m|D) ∝ P(m) · Ẑ. Model selection comes free with the audit. No extra machinery, no extra sampling.

The test that matters: I generated data from delayed adstock peaking at lag 2, then made the fake network confidently wrong — 90% sure the family was geometric. The data answered with a log Bayes factor of +20 in favor of delayed (in evidence terms, overwhelming), and the corrected posterior landed on the true family at probability 1.000 anyway. The winning family then recovered the peak lag as 2.08 ± 0.15 against a truth of 2.0.

The network’s opinion, like everything else it produces, is advisory. No prior confidence survives twenty nats of evidence.

Result 3: the meta-learner is real, calibrated, and passes its own gate

Prototypes 1 and 2 used stand-in proposals. Prototype 3 replaces them with an actual trained network — and to make the point that this requires no exotic infrastructure, I implemented the neural posterior estimator in pure numpy, with manual backpropagation, gradient-checked to eight decimal places. It runs anywhere Python runs.

The pipeline: 10,000 marketing panels simulated from the prior → 33 summary statistics per panel (cross-correlograms between spend and revenue at various lags, autocorrelations, adstock probes) → a small MLP that outputs a full-covariance Gaussian posterior via a Cholesky head. Training takes minutes on a laptop.

Three results, in ascending order of importance:

It is calibrated out of the box. Calibration means the network’s confidence is honest: when it says “80% sure the answer is in this range,” the answer should be in that range 80% of the time. On 1,000 held-out simulated panels, its 80% intervals covered the truth at 0.808, 0.842, 0.804, and 0.799 across the four structural parameters — essentially perfect. This is the simulation-based calibration check that the design document mandates as an automated CI gate (a network that miscalibrates should be a failed build, not a client incident), and a freshly trained network passes it.

It discovered the identifiability trade-off by itself. Fed a panel it had never seen, the network’s output covariance carried corr(log k, log β) = +0.93 — matching the oracle answer from hours of MCMC, exactly. Nobody told it about the saturation-scale trade-off. From summary statistics alone, it inferred that this dataset cannot separately identify those two parameters, and widened its joint uncertainty in precisely the right direction. For a system whose uncertainty statements feed budget decisions, this is the property that matters most: it knows what the data doesn’t know.

It passes the gate. The trained network’s proposal on the unseen panel scored ESS/n = 0.319 — ACCEPT. Exact Bayesian posterior, neural-network speed, and not one knob turned by a human.

Left: held-out coverage of 80% intervals, target 0.80. Right: the correlation structure the network recovered — including the 0.93 identifiability trade-off nobody told it about

What this does to the Vector1 stack

These results aren’t a standalone trick — they reorganize the roles of the tools I’ve spent the last years building, and each one comes out stronger:

● Papilon gains its 3.0 thesis: a zero-configuration inference path. Today Papilon packages the Bayesian MMM pipeline, the budget optimizer, and the agentic orchestration layer — but a human still configures it. With the gated meta-learner in front, Papilon becomes point-at-data infrastructure: panel in, calibrated and audited posterior out, budget optimization downstream, and a trust scalar attached to every number. The remaining human inputs are objectives, constraints, and risk tolerance — the knobs that encode business judgment and should stay human.

● PyCausalSim is promoted from testing tool to teacher. It was built to stress-test causal assumptions by simulating structural models under interventions. This work reveals its bigger job: PyCausalSim’s generative families are the prior the meta-learner trains on. Every structural family it can express is a pattern the network can learn to recognize; the simulator’s breadth directly becomes the system’s competence. The same engine then doubles as the evaluation gym — synthetic worlds with known truth where the trained network is scored before it ever touches a real budget.

● Memory-Node Encapsulation (MNE), my research on artificial episodic memory, provides the loop that makes the whole system compound. Every time the gate escalates, that’s a documented encounter with a marketing environment the simulator couldn’t express — and it’s logged as an episode: the panel, the failed proposal, the diagnostics, and eventually the resolved truth. Those episodes expand PyCausalSim’s families and retrain the network. The sim-to-real gap — the standard fatal objection to simulation-trained systems — becomes a monitored, shrinking quantity with a paper trail. The system doesn’t just work; it accumulates.

That composition — simulator as teacher, network as fast expert, classical statistics as auditor, episodic memory as the flywheel — is the Vector1 research program in one sentence. None of the three tools was designed with the others’ new role in mind, which is exactly why the fit is convincing: the architecture found them, not the reverse.

The honest caveats, and what’s next

Everything above runs on a toy model where simulator and reality coincide by construction. The prototypes prove the architecture — the trust contract, the free model selection, the learnability and calibration of the proposals. They do not yet prove survival against real panels, real confounding, real data pathology. That’s the next claim to earn, and the path is engineering rather than concept: the 33 summary statistics become a proper factorized panel encoder (permutation-invariant across geos, patch-based across time), the Gaussian head becomes a normalizing flow, and the toy simulator becomes PyCausalSim’s full domain-randomized mixture of structural families. The paper specifies all three, alongside the calibration harness that should gate every retrain.

The larger point stands on the results already in hand. The knobs were never the expertise. The expertise was knowing what the knobs meant — and that, it turns out, is learnable, auditable, and free to check.

Paper: Amortized Structural Inference for Media Mix Models: A Gated Architecture for Calibrated, Zero-Configuration Measurement (Curry, 2026). Code: github.com/Bodhi8/papilon.

About the Author

Brian Curry is a Kansas City–based AI researcher, data scientist, and the founder of Vector1 Research, where he works on the intersection of cognitive AI architecture, causal inference, and economic systems. His research introduced Memory-Node Encapsulation (MNE), an original data structure for artificial episodic memory, and the NeoCortex-M architecture for memory-driven AI. He is the creator of Papilon, an open-source framework for complex system optimization with agentic orchestration, PyCausalSim, a Python framework for causal discovery through simulation, and Daedalus, a platform for causal and economic analysis.

His practitioner work focuses on building production intelligence systems — causal marketing analytics, agentic systems, and the serving infrastructure that makes them deployable inside enterprise environments. He writes about that work at vector1.ai and on Medium.

Connect: brian@vector1.ai