Informative Prior Distributions#

Part 1 · Stage 2 · 📍 Single-Parameter Models & Priors · Lesson 014 of 144 · beginner

◀ Previous · Summarizing Posterior Inference · Next · Normal Distribution with Known Variance ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Using what you know#

An informative prior deliberately encodes substantive knowledge about a parameter before the current data arrive — from previous studies, physical constraints, or expert judgement. Its purpose is not to bias the answer toward a preferred conclusion but to stop the analysis from pretending to know nothing when, in fact, a great deal is known.

Priors as pseudo-data#

Conjugate priors make “how much” precise: their hyperparameters read as prior observations. In the Beta–Binomial, \(\mathrm{Beta}(\alpha, \beta)\) acts like \(\alpha\) prior successes and \(\beta\) prior failures, an effective prior sample size of \(\alpha + \beta\). In the Poisson–Gamma, \(\mathrm{Gamma}(\alpha, \beta)\) acts like \(\alpha\) pseudo-events observed in \(\beta\) pseudo-exposure. This is the honest way to calibrate an informative prior: ask how many observations is my belief worth?

from scipy import stats
# "I've seen roughly 20 conversions in 100 visits before" -> Beta(20, 80)
prior = stats.beta(20, 80)
prior.mean(), prior.std()            # 0.20, ~0.04   (prior sample size 100)
# 5 conversions in 40 new visits:
post = stats.beta(20 + 5, 80 + 35)   # exact conjugate update
post.mean()                          # 0.179 — pulled from 0.125 toward 0.20

Eliciting a prior#

Elicitation usually works backwards from summaries an expert can state: a plausible central value and a range they would be surprised to see violated. Match those to a distribution’s mean and spread, then check the implications — draw from the prior, simulate data (a prior predictive check), and ask whether the simulated datasets look like ones the world could produce. A prior that generates absurd data is wrong, however reasonable it looked as a density.

The discipline#

Informative priors carry real risk: with sparse data, they can dominate, and a confidently wrong prior yields a confidently wrong posterior. Three habits keep them honest. State the prior and its justification explicitly. Report its effective sample size, so readers see how much it contributes. And run a sensitivity analysis — refit under reasonable alternatives, and say so when conclusions move. Between the extremes of an informative prior and a deliberately vague one lies the weakly informative prior of the coming lessons.

Hint