Multiple imputation#

Part 4 · Stage 14 · 🛡️ Robustness & Missing Data · Lesson 119 of 144 · advanced

◀ Previous · Notation · Next · Missing data in the multivariate normal and t models ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Fill in the blanks, honestly#

The obvious fix for a missing value — plug in a single best guess (the mean, a regression prediction) — is quietly wrong: it treats an estimate as if it were known, so every downstream standard error comes out too small. Multiple imputation repairs this by filling the gaps not once but many times, carrying the uncertainty about the missing values all the way through to the final answer.

The three steps#

Impute. Draw \(m\) complete datasets, each with the missing entries replaced by values sampled from their posterior predictive distribution given the observed data — genuine draws, not point estimates, so the imputations differ from one another.
Analyse. Run the intended analysis on each completed dataset separately, giving \(m\) estimates \(\hat{Q}_k\) and their variances \(U_k\).
Pool. Combine them with Rubin’s rules.

\[\bar{Q} = \frac{1}{m}\sum_k \hat{Q}_k, \qquad \bar{U} = \frac{1}{m}\sum_k U_k, \qquad B = \frac{1}{m-1}\sum_k (\hat{Q}_k - \bar{Q})^2,\]

\[T = \bar{U} + \Bigl(1 + \tfrac{1}{m}\Bigr) B .\]

The total variance \(T\) is the heart of it. \(\bar{U}\) is the ordinary within-imputation uncertainty; \(B\), the between-imputation variance, measures how much the answer wobbles as the imputations change — and that term is exactly the extra uncertainty due to missingness that single imputation throws away.

import numpy as np
Qk = np.array([analyze(dataset) for dataset in imputed_datasets])   # m estimates
Uk = np.array([variance(dataset) for dataset in imputed_datasets])  # m variances
Q_bar = Qk.mean()
U_bar = Uk.mean()
B = Qk.var(ddof=1)                                                  # between-imputation
T = U_bar + (1 + 1/len(Qk)) * B                                     # total variance
se = np.sqrt(T)

The Bayesian reading#

Multiple imputation is a Bayesian computation in disguise: the missing values are unknown parameters, and the \(m\) completed datasets are draws from their posterior. In a fully Bayesian model you get this for free — sample \(y_{\text{mis}}\) jointly with \(\theta\) in one MCMC run, and the posterior already integrates over the missing values with no separate pooling step. Rubin’s rules are what you use when the analysis model and the imputation model are separate programs; a joint Bayesian model makes them one.

Practicalities#

A few settled points. Classic advice said \(m = 5\); modern practice prefers twenty or more, since it is cheap and stabilises \(B\). The imputation model must be at least as rich as the analysis model — if you will study an interaction, the imputation must include it, or the imputations will erase it. And it all rests on MAR: multiple imputation handles ignorable missingness, and MNAR still demands an explicit model. Within those limits it is the standard, principled way to keep incomplete data from silently understating what you do not know.

Hint