📈 Bayesian Time Series

Bayesian Time Series#

Time-series modelling in a Bayesian framework, yielding full posterior uncertainty over parameters and forecasts.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Why go Bayesian for time series#

Classical models — ARIMA, exponential smoothing — fix their parameters and estimate them by maximum likelihood, returning a single point forecast. Real forecasting often needs more: uncertainty quantification (a full distribution of future values, not one number), flexibility (priors encoding trend, smoothness, seasonality, and handling missing data), and online updating (re-forecasting as new data lands). Bayesian methods deliver all three through posterior distributions.

The general framework#

Model a series \(y_{1:T}\) with latent parameters \(\theta\):

\[p(\theta \mid y_{1:T}) \propto p(y_{1:T} \mid \theta)\, p(\theta),\]

where the prior \(p(\theta)\) encodes structure (smoothness, trend, seasonality) and the likelihood \(p(y_{1:T} \mid \theta)\) is often Gaussian or Poisson. Forecasts come from the posterior predictive, which integrates over parameter uncertainty rather than plugging in a point estimate:

\[p(y_{T+h} \mid y_{1:T}) = \int p(y_{T+h} \mid \theta, y_{1:T})\, p(\theta \mid y_{1:T})\, d\theta.\]

The model families#

Bayesian ARIMA/ARMA — ARIMA with priors on \(\phi, \theta, \sigma^2\); inference by MCMC or variational methods, yielding posterior forecasts.
State-space models / dynamic linear models (DLMs) — a hidden state evolves and emits observations,

\[x_t = F x_{t-1} + w_t, \qquad y_t = H x_t + v_t,\]

covering local-level and trend-plus-seasonality decompositions; inference via the Kalman filter (conjugate Gaussian) or a particle filter (nonlinear / non-Gaussian).
Bayesian structural time series (BSTS) — decomposes into trend + seasonality + regressors + holiday effects; the engine behind Google’s CausalImpact for estimating intervention effects (A/B tests, policy evaluation).
Gaussian-process time series — a GP prior on the latent function, \(f \sim \mathcal{GP}(0, k(t,t'))\), with the kernel \(k\) encoding smoothness and periodicity; flexible but heavy on long series.
Bayesian VAR — multivariate autoregression with shrinkage (Minnesota) priors on the coefficient matrices to tame overfitting.

Inference toolkit#

Conjugate Gibbs sampling (Normal–Inverse-Gamma for AR models), Hamiltonian Monte Carlo (Stan, PyMC), variational inference for speed, and particle MCMC for nonlinear state spaces.

Trade-offs#

The payoff is full posterior predictive distributions, uncertainty over both parameters and future values, principled priors for domain knowledge, and graceful handling of missing data and regime shifts. The cost is computation (MCMC on large data), the need for careful prior choice, and scaling limits (GPs on long series).

Theme: Signal Processing & Time Series · All terminology

Hint

Mind map — connected ideas

Time Series · Bayesian Inference. · Posterior · Bayesian Decision Theory (BDT) · Bayesian Neural Networks (BNNs)

Hint

More in Signal Processing & Time Series

ARIMA (AutoRegressive Integrated Moving Average) · Forecast Error · Forecasting Benchmarks · Forecasting Competitions · Log-Space · Low-pass Filtering · LSTM — Long Short-Term Memory Networks · M-Competitions (Makridakis Competitions) · Naïve Baseline Forecast · Prophet — Time Series Forecasting by Facebook (Meta) · Seasonal Lag · Seasonality · Signal Processing · Simple Baseline Methods