Preliminary Estimation for AR Models and the Yule–Walker Equations#

Stage 5 · 🧮 Estimation · Lesson 11 of 18 · advanced

◀ Previous · Sample ACF and Sample PACF · Next · Maximum Likelihood Estimation for ARMA Models (Gaussian MLE) ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

The idea#

The Yule–Walker method is the simplest way to fit an AR(p) model: it is method of moments — match the model’s theoretical autocovariances to the ones you measure, and solve for the coefficients. No optimisation, no starting values.

The equations#

Multiplying the AR recursion by lagged values and taking expectations links the coefficients to the autocovariances through a Toeplitz linear system:

\[\Gamma_p\, \boldsymbol{\phi} = \boldsymbol{\gamma}_p, \qquad \Gamma_p = \big[\gamma(i-j)\big]_{i,j=1}^{p}, \quad \boldsymbol{\gamma}_p = \big(\gamma(1), \dots, \gamma(p)\big)^{\!\top}.\]

Because \(\Gamma_p\) is a full-rank symmetric Toeplitz matrix, the solution \(\boldsymbol{\phi} = \Gamma_p^{-1}\boldsymbol{\gamma}_p\) always exists.

Plugging in the data#

Replace each \(\gamma(\cdot)\) with its sample counterpart \(\hat{\gamma}(\cdot)\) to get \(\hat{\boldsymbol{\phi}} = \hat{\Gamma}_p^{-1}\hat{\boldsymbol{\gamma}}_p\), and read off the noise variance as \(\hat{\sigma}_w^2 = \hat{\gamma}(0) - \hat{\boldsymbol{\phi}}^{\!\top}\hat{\boldsymbol{\gamma}}_p\). The Durbin–Levinson recursion solves the system without an explicit inverse; in statsmodels this is yule_walker.

Why it’s a good start#

For a genuine AR(p), Yule–Walker estimators are consistent, asymptotically normal, and as efficient as maximum likelihood — and they always return a causal (stationary) model. That reliability makes them the standard preliminary estimate and a natural starting point for the likelihood methods below. (For ARMA the same idea works but is no longer efficient, so it is used only to seed the optimiser.)

Hint