🔁 Bayes' Theorem

Bayes’ Theorem#

The rule that turns a prior into a posterior using the likelihood: posterior is proportional to likelihood x prior.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Bayes’ theorem is the rule for updating a belief when new evidence arrives. It is the single equation that ties together the four quantities of Bayesian inference — prior, likelihood, evidence and posterior.

The formula#

For a hypothesis \(H\) and data \(D\),

\[P(H \mid D) = \frac{P(D \mid H)\, P(H)}{P(D)},\]

where \(P(H)\) is the prior, \(P(D \mid H)\) the likelihood, \(P(D)\) the marginal likelihood (evidence / normaliser) and \(P(H \mid D)\) the posterior. Stripped to its working form,

\[\text{Posterior} \propto \text{Prior} \times \text{Likelihood}.\]

Worked example — the base-rate trap#

A disease affects 1% of people; a test is 99% sensitive and has a 5% false-positive rate. A patient tests positive — how likely are they to be sick? The evidence is

\[P(+) = \underbrace{0.99 \times 0.01}_{\text{true positive}} + \underbrace{0.05 \times 0.99}_{\text{false positive}} = 0.0099 + 0.0495 = 0.0594,\]

so

\[P(\text{disease} \mid +) = \frac{0.0099}{0.0594} \approx 0.167.\]

Despite a positive result on a “99% accurate” test, the chance of disease is only about 16.7% — because the disease is rare, false positives swamp the true ones. This base-rate fallacy is exactly what Bayes’ theorem corrects: the rare prior pulls the posterior far below the test’s sensitivity.

Where it shows up#

Bayesian inference and updating, the Naive Bayes classifier and Bayesian networks, medical diagnostics, A/B testing (Bayesian sequential testing, posterior probability of uplift), fraud and risk estimation, and everyday belief revision under new information.

Theme: Bayesian Inference · All terminology

Hint

Mind map — connected ideas

Prior Belief (or Prior Probability) · Posterior · Marginal Likelihood (also called The Model Evidence or Integrated Likelihood) · Binomial Likelihood · Bayesian Inference. · Posterior Probability

Hint

More in Bayesian Inference

Bayesian Correction · Bayesian Decision Theory (BDT) · Bayesian Inference. · Bayesian Neural Networks (BNNs) · Binomial Likelihood · Gaussian Processes (GPs) · Marginal Likelihood (also called The Model Evidence or Integrated Likelihood) · MCMC (Markov Chain Monte Carlo) · Parameter(s) of Interest · Posterior · Posterior belief · Posterior Probability · Posterior probability of uplift · Prior Belief (or Prior Probability)