Posterior as a Compromise Between Data and Prior Information#

Part 1 · Stage 2 · 📍 Single-Parameter Models & Priors · Lesson 012 of 144 · beginner

◀ Previous · Estimating a Probability from Binomial Data · Next · Summarizing Posterior Inference ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Between two answers#

The posterior always lies between the prior and the data. The Beta–Binomial makes this exact rather than metaphorical: the posterior mean is a weighted average of the prior mean and the sample proportion, and the weights are interpretable.

The weighted average#

With \(\theta \mid y \sim \mathrm{Beta}(\alpha + y,\ \beta + n - y)\), the posterior mean is

\[\mathrm{E}[\theta \mid y] = \frac{\alpha + y}{\alpha + \beta + n} = \underbrace{\frac{\alpha + \beta}{\alpha + \beta + n}}_{\text{prior weight}} \cdot \frac{\alpha}{\alpha + \beta} \;+\; \underbrace{\frac{n}{\alpha + \beta + n}}_{\text{data weight}} \cdot \frac{y}{n} .\]

The prior mean is \(\alpha / (\alpha+\beta)\); the data’s answer is the sample proportion \(y/n\). Each is weighted by its effective sample size — \(\alpha + \beta\) for the prior, \(n\) for the data. A \(\mathrm{Beta}(2, 8)\) prior carries the weight of ten prior observations; against \(n = 10\) it contributes about half the answer, against \(n = 200\), about five per cent.

The data wins, eventually#

The data weight \(n / (n + \alpha + \beta)\) climbs to 1 as \(n \to \infty\). So the posterior mean converges to the sample proportion, and the posterior itself concentrates — a preview of the large-sample theory in Stage 4. Two analysts with different (reasonable) priors are driven to agreement by enough data. The prior matters most exactly where it should: when data are sparse.

from scipy import stats
for n in (10, 100, 1000):
    y = int(0.8 * n)                       # same proportion, growing n
    a, b = 2, 8                            # prior mean 0.2, prior "sample size" 10
    print(n, round(stats.beta(a + y, b + n - y).mean(), 3))
# 10 -> 0.500   100 -> 0.745   1000 -> 0.792   (approaching 0.8)

Shrinkage, and its price#

Pulling the estimate toward the prior mean is shrinkage, and it is a feature: it regularises noisy small-sample estimates and prevents the absurdity of a 0% or 100% rate from three trials. The price is that a badly chosen informative prior biases the answer, most damagingly when \(n\) is small and the pull is strongest. Hence the discipline: state the prior, justify it, and check the sensitivity of conclusions to reasonable alternatives.

Hint