Building ML Models That Survive Regime Changes

Machine learning models trained on financial data face a unique challenge: the data-generating process is non-stationary. Market regimes — volatility levels, correlation structures, liquidity conditions — shift abruptly, and a model trained on one regime often performs poorly in the next. This is the primary reason that impressive backtest results frequently fail to translate into live performance.

At ShoalFlow, we have developed three architectural patterns that improve model robustness across regime changes. The first is regime-aware ensembling: rather than training a single model on all available data, we train multiple specialist models on distinct regime clusters (identified by unsupervised methods applied to volatility, dispersion, and correlation features) and route live predictions through a meta-model that estimates the current regime and weights specialists accordingly.

The second pattern is continuous retraining with decay weighting. Instead of retraining on a fixed lookback window, we retrain daily using all available history but apply an exponential decay function that gives recent observations higher weight. This allows the model to adapt to drifting distributions without discarding the structural patterns present in older data.

The third pattern is explicit uncertainty quantification. Every prediction our models produce comes with a calibrated confidence interval. When confidence drops below a configurable threshold — typically a sign that current conditions are outside the training distribution — the model automatically reduces position sizing or defers to a simpler fallback strategy. This "know what you don't know" approach prevents the catastrophic losses that occur when a model confidently extrapolates into unfamiliar territory.