Maximum Likelihood Estimation for ARMA Models (Gaussian MLE)#

Stage 5 · 🧮 Estimation · Lesson 12 of 18 · advanced

◀ Previous · Preliminary Estimation for AR Models and the Yule–Walker Equations · Next · Diagnostics After Fitting a Time Series Model ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

The likelihood#

If the ARMA process is driven by Gaussian white noise, then the whole sample \((x_1, \dots, x_n)\) is multivariate normal with a covariance matrix determined by the parameters. Maximum likelihood picks the \((\boldsymbol{\phi}, \boldsymbol{\theta}, \sigma_w^2)\) that make the observed data most probable under that model.

The innovations trick#

Working with the raw \(n \times n\) covariance is expensive. The standard route rewrites the likelihood through the one-step prediction errors (“innovations”) and their variances — computed cheaply by the innovations algorithm or a Kalman filter. This turns the likelihood into a product over time and avoids inverting a large matrix.

A harder optimisation#

Unlike Yule–Walker, there is no closed form — the likelihood is nonlinear, especially in the moving-average parameters, so it must be maximised numerically. That optimisation can be finicky, which is exactly why good starting values (from Yule–Walker or the Hannan–Rissanen method) matter. One further choice: the conditional likelihood fixes the initial values, while the exact likelihood models them too.

Why it’s the default#

Maximum likelihood is consistent, asymptotically normal, and efficient — the lowest-variance estimates available — and, remarkably, its asymptotic distribution is the same even when the data are not Gaussian. That combination is why statsmodels’ ARIMA fits by exact MLE (via a state-space / Kalman filter) by default.

Hint