Forecasting Models That Actually Work in Production: ARIMA vs Prophet vs Bayesian (PyMC)

Every data science team eventually needs to forecast something — sales, demand, headcount, server load. And every team eventually discovers the same painful truth: the model that wins on Kaggle usually fails in production.

I’ve deployed forecasting systems across multiple companies — from workforce planning automation with Prophet to Bayesian structural models predicting daily comparable sales across 5 European markets. This post covers what actually works, what doesn’t, and why Bayesian decomposition changed the game.

— -

The Forecasting Landscape

There are four main families you’ll encounter in production:

• Statistical (ARIMA, ETS): Mathematically elegant, but rigid. Parameter selection is manual and painful at scale.

• Decomposition (Prophet, STL): Break series into trend, seasonality, and events. Production-friendly and interpretable.

• Machine Learning (XGBoost, LightGBM): Often wins on accuracy but loses on interpretability.

• Bayesian Structural (PyMC, Stan): Encode domain knowledge as priors, decompose effects explicitly, quantify uncertainty natively. The production powerhouse when causal reasoning matters.

— -

Prophet: The 80/20 Solution

Prophet’s additive decomposition — y(t) = g(t) + s(t) + h(t) + ε(t) — is what makes it production-friendly. Business stakeholders understand “trend is up, weekly patterns exist, and Thanksgiving causes a spike.” Try explaining ARIMA(2,1,1) to a VP.

For workforce forecasting, the practical wins were:

• Handles missing data gracefully (holidays, system outages)

• Multiple seasonality via Fourier components

• Automatic changepoint detection (critical during demand shifts)

• External regressors for promotions and events

Combined with Bayesian hyperparameter optimization (Optuna), this reduced manual planning effort by 96%.

— -

Bayesian Sales Forecasting at Scale

The most technically ambitious system I’ve led was a Bayesian daily sales forecasting platform predicting comparable sales across 5 European markets.

Why Not Prophet This Time?

The problem demanded explicit causal decomposition of promotion effects with cannibalization modeling. When a restaurant runs a discount deal, it drives traffic to a specific menu segment — but also cannibalizes baseline sales. We needed a model that could separately estimate both effects and let business users adjust individual assumptions.

The Model

We chose PyMC with MAP inference:

log(sales) = α_segment # per-segment intercepts

• β_slope · t # per-segment trends
• β_seasonality · S(t) # Fourier seasonality
• β_holiday · H(t) # holiday effects
• β_act_segment · A(t) # activation → segment lift
• β_act_baseline · B(t) # activation → baseline cannibalization
• ε

Each component has a clear business interpretation:

• Activation-to-segment coefficients measure the promotion lift
• Activation-to-baseline coefficients measure cannibalization
• Net incrementality per promotion = combining both, not just gross lift

Architecture

The system runs on Snowflake as the data platform, with:

• Feature storage for daily comparable sales, activation calendars, and holiday calendars
• Per-country PyMC MAP models with Fourier seasonality and dual-headed activation effects
• Model coefficients, forecasts, and estimated activation tables stored as versioned Snowflake tables
• A multi-page Streamlit dashboard with: — Main Dashboard for sales forecast visualization — Activations Deep Dive for per-promotion effect analysis — Model Coefficients page where marketing teams can adjust activation strength and incrementality estimates

Key Technical Decisions

MAP instead of full MCMC. With 5 countries and dozens of activations each, full posterior sampling was too slow for weekly retraining. MAP gave us point estimates with prior regularization — fast enough, principled enough.

User-adjustable coefficients. Marketing teams can override activation strength for future promotions via the Streamlit dashboard. Adjustments are versioned and automatically consolidated with model outputs on the next training run.

Strict schema validation. Every data handoff uses Pandera schemas. When activation calendars changed upstream, we caught it immediately instead of producing garbage forecasts.

The Activation Coefficient Pipeline

The most intricate subsystem:

1. PyMC outputs raw coefficients: act__{type}__{activation_name}
2. Pipeline parses, pivots by coefficient type (activation_to_baseline vs activation_to_segment), merges with activation calendar for date ranges
3. Business metrics derived: Estimated Activation Strength = exp(segment effect), Estimated Incrementality = (1 + baseline effect) × 100
4. User adjustments layered via merge-edit-delete operations, stored as versioned history
5. Next model run reconciles model estimates with user overrides

Results

23% improvement in forecast accuracy over the previous approach, with same-store sales decomposition used directly in monthly financial reporting. Marketing teams went from ignoring model outputs to actively using them for promotion planning.

— -

When to Use What

| Approach | Best For | | — — — — — | — — — — — | | ARIMA/ETS | Single univariate series, benchmarking | | Prophet |Multiple seasonalities, missing data, dozens of series | | Bayesian (PyMC/Stan) | Causal decomposition, cannibalization, user-adjustable assumptions | | ML (XGBoost) | Rich features, cross-series learning, accuracy focus | | Deep Learning | Thousands of related series, massive data |

— -

Production Pitfalls to Avoid

1. Overfitting. Always use time-based CV, never random splits.
2. Ignoring uncertainty. Point forecasts without confidence intervals are dangerous.
3. Not monitoring drift. Set automated alerts when rolling error exceeds a threshold.
4. Forecasting too far ahead. Accuracy degrades exponentially with horizon.
5. Ignoring cannibalization. If your model can’t decompose gross lift from net incrementality, you’re overstating promotion ROI.

— -

Conclusion

Production constraints dominate model choice. The best forecasting model isn’t the one with the lowest test set error — it’s the one that handles your data’s messiness, scales to your number of series, produces interpretable results, and runs reliably without manual intervention.

For most problems, Prophet with Bayesian hyperparameter tuning is the sweet spot. But when you need causal decomposition and user-adjustable assumptions, PyMC is worth the engineering investment. It’s the difference between a model that predicts and a system that stakeholders actually use to make decisions.

— -

Originally published at abhilashganji.com