Observational studies#

Part 2 · Stage 7 · 🗳️ Data Collection & Decisions · Lesson 055 of 144 · intermediate

◀ Previous · Sensitivity and the role of randomization · Next · Censoring and truncation ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

When you cannot randomise#

Most questions cannot be settled by an experiment: you cannot randomise people to smoke, to be poor, or to live near a highway. In an observational study the treatment was chosen — by the units, by doctors, by circumstance — and the assignment mechanism is unknown. Everything then depends on whether ignorability can be recovered by conditioning.

Ignorability conditional on covariates#

The working assumption is that treatment is as good as random given the observed covariates:

\[\Pr\bigl(W \mid y(0), y(1), X\bigr) = \Pr(W \mid X) \qquad \text{(``no unmeasured confounding'')}.\]

This is the MAR condition transposed to causal inference. Its opposite — assignment depending on the unobserved potential outcomes — is the MNAR case, and it is not testable: two datasets identical in \((y_{\text{obs}}, W, X)\) are consistent with both. No amount of data settles it; only knowledge of why treatment was assigned can.

Two routes to conditioning#

Given the assumption, the estimand is recovered by conditioning on \(X\) — either by modelling the outcome \(p(y \mid W, X)\) and averaging its predictions over the covariate distribution, or by modelling the propensity \(e(x_i) = \Pr(W_i = 1 \mid x_i)\) and comparing units of like propensity.

import pymc as pm
with pm.Model():
    beta = pm.Normal("beta", 0, 2.5, shape=X.shape[1])
    tau  = pm.Normal("tau", 0, 1)
    pm.Normal("y", X @ beta + tau * W, pm.HalfNormal("s", 1), observed=y)
    idata = pm.sample()
# ATE = posterior mean of predictions with W=1 minus W=0, averaged over the sample

Bayesian practice tends to favour the outcome model — it uses the posterior directly, propagates uncertainty, and permits hierarchical structure — while treating the propensity as a diagnostic: where the treated and untreated propensity distributions barely overlap, the comparison is an extrapolation, and no model can rescue it.

Which covariates?#

Include the variables that affect assignment, and the pre-treatment variables that affect the outcome. But conditioning is not universally benign: adjusting for a variable on the causal pathway between treatment and outcome removes part of the effect you want, and adjusting for a collider can create bias where none existed. Covariate choice requires causal reasoning about the mechanism, not a search over predictors — and the final report should say, plainly, how strong an unmeasured confounder would need to be to change the conclusion.

Hint