🔁 Prior Belief (or Prior Probability)

Prior Belief (or Prior Probability)#

Belief about a parameter before observing the current data.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

A prior belief (or prior probability) is your initial belief about a parameter before seeing any new data, encoded as a prior distribution. It captures what is plausible from past studies, domain expertise or reasonable assumptions: what you think before you see the evidence.

Where it sits#

In Bayes’ theorem the prior \(P(H)\) is the term that the likelihood multiplies and the data update:

\[P(H \mid D) = \frac{P(D \mid H)\, P(H)}{P(D)}.\]

Kinds of prior#

Informative — built on strong prior knowledge (a drug’s effect is likely 5–10%, from past trials).
Non-informative / flat — expresses near-ignorance (every value equally likely, e.g. \(\text{Beta}(1,1)\)).
Weakly informative — adds gentle, realistic bounds to rule out absurd values (conversion rates almost never exceed 50%), which stabilises inference without forcing a conclusion.

Example — coin toss#

For the probability of heads \(p\): a dogmatic “fair coin” prior fixes \(p = 0.5\); total uncertainty is \(\text{Beta}(1,1)\) (uniform); “probably fair but unsure” is \(\text{Beta}(20, 20)\), peaked at 0.5. Each is updated by the data into a posterior.

In A/B testing#

A sensible prior on the conversion-rate difference — centred at 0 with small variance, reflecting that new features rarely move conversion more than a few points — keeps early results from over-reacting to noise; the experiment then updates it into a posterior on the uplift.

The key caveat#

The choice of prior matters most with small datasets and fades as data grows (the likelihood takes over). A weakly informative prior is usually the safe default: enough structure to regularise, not so much that it overrides the evidence.

Theme: Bayesian Inference · All terminology

Hint

Mind map — connected ideas

Posterior · Bayes’ Theorem · Beta Distribution · Posterior belief · Binomial Likelihood

Hint

More in Bayesian Inference

Bayes’ Theorem · Bayesian Correction · Bayesian Decision Theory (BDT) · Bayesian Inference. · Bayesian Neural Networks (BNNs) · Binomial Likelihood · Gaussian Processes (GPs) · Marginal Likelihood (also called The Model Evidence or Integrated Likelihood) · MCMC (Markov Chain Monte Carlo) · Parameter(s) of Interest · Posterior · Posterior belief · Posterior Probability · Posterior probability of uplift