🔁 Binomial Likelihood

Binomial Likelihood#

The probability model for the number of successes in a fixed number of independent Bernoulli trials.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

The setup#

Model \(n\) independent Bernoulli (success/failure) trials with success probability \(p\), observing \(k\) successes. The binomial PMF gives the probability of exactly \(k\) successes:

\[P(X = k \mid p) = \binom{n}{k} p^k (1 - p)^{n - k}.\]

Likelihood vs PMF#

The likelihood is the same expression read the other way: with the data \((k, n)\) fixed, it is a function of the unknown \(p\),

\[L(p \mid k, n) = \binom{n}{k} p^k (1 - p)^{n - k} \;\propto\; p^k (1 - p)^{n - k},\]

dropping the binomial coefficient because it doesn’t depend on \(p\). The PMF asks “given \(p\), how probable is this data?”; the likelihood asks “given this data, how well does each \(p\) explain it?”.

Maximum likelihood#

Maximising the log-likelihood,

\[\ell(p) = k \ln p + (n - k)\ln(1 - p),\]

and setting the derivative to zero gives the intuitive estimate

\[\hat{p}_{\text{MLE}} = \frac{k}{n},\]

the observed success proportion. For \(n = 10,\ k = 7\), the likelihood \(\propto p^7 (1-p)^3\) peaks at \(\hat{p} = 0.7\).

The Bayesian half#

Pair the binomial likelihood with a Beta prior and conjugacy makes the posterior Beta as well:

\[\text{Beta}(\alpha, \beta) \times \text{Binomial}(k, n) \;\Rightarrow\; \text{Beta}(\alpha + k,\; \beta + n - k).\]

So the binomial likelihood sits at the centre of both worlds — its MLE is the frequentist sample proportion, and with a Beta prior it gives the classic Beta–Binomial Bayesian model.

Theme: Bayesian Inference · All terminology

Hint

Mind map — connected ideas

Beta Distribution · Posterior belief · Bayes’ Theorem · Bayesian Inference. · Frequentist

Hint

More in Bayesian Inference

Bayes’ Theorem · Bayesian Correction · Bayesian Decision Theory (BDT) · Bayesian Inference. · Bayesian Neural Networks (BNNs) · Gaussian Processes (GPs) · Marginal Likelihood (also called The Model Evidence or Integrated Likelihood) · MCMC (Markov Chain Monte Carlo) · Parameter(s) of Interest · Posterior · Posterior belief · Posterior Probability · Posterior probability of uplift · Prior Belief (or Prior Probability)