Bayesian Inference#

Part 1 · Stage 1 · 🎲 The Bayesian Idea · Lesson 003 of 144 · beginner

◀ Previous · General Notation for Statistical Inference · Next · Discrete Bayesian Examples – Genetics and Spell Checking (with θ) ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Conclusions as probabilities#

Bayesian inference is the process of drawing conclusions about unknown quantities as probability statements conditional on the observed data. Not “the estimate is 0.58 ± 0.05”, but “given these data, there is a 93% probability that \(\theta\) exceeds 0.5”. Every conclusion is read off the posterior distribution, and every conclusion carries its uncertainty with it.

Bayes’ rule, again#

The machinery is one line. Starting from the joint model \(p(\theta, y) = p(\theta) p(y \mid \theta)\) and conditioning on \(y\):

\[p(\theta \mid y) = \frac{p(\theta)\, p(y \mid \theta)}{p(y)}, \qquad p(y) = \int p(\theta)\, p(y \mid \theta) \, d\theta .\]

The denominator — the marginal likelihood or evidence — does not depend on \(\theta\), so for inference it merely normalises. This is why the unnormalised form \(p(\theta \mid y) \propto p(y \mid \theta)\, p(\theta)\) is the working equation, and why samplers need only the numerator.

Reading a posterior#

Once you have the posterior (analytically or as samples), every question is answered by summarising it:

import numpy as np
post = ...                                   # draws from p(theta | y)
post.mean(), np.median(post)                 # point summaries
np.percentile(post, [2.5, 97.5])             # 95% credible interval
(post > 0.5).mean()                          # P(theta > 0.5 | y), directly

That last line is the Bayesian signature: a probability of a hypothesis, computed by counting draws.

What differs, and why#

Bayesian and frequentist conclusions often agree in simple problems with plenty of data. They diverge where conditioning matters: small samples, many parameters, hierarchical structure, or genuine prior information. The cost is that you must state a prior; the benefit is that the answer is a distribution you may interpret directly, and that uncertainty propagates automatically into any derived quantity.

Hint