🎲 Likelihood

Likelihood#

How probable observed data are under a model’s parameters.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

The likelihood is the probability of the observed data given a model’s parameters — \(\mathcal{L}(\theta) = P(\text{data} \mid \theta)\). The twist is perspective: it is read as a function of the parameters \(\theta\) (with the data fixed), asking which parameters make what we saw most probable?

Maximum likelihood#

MLE picks the parameters that maximize the likelihood (in practice the log-likelihood, since sums are easier and more stable than products):

\[\hat{\theta}_{\text{MLE}} = \arg\max_{\theta} \; \mathcal{L}(\theta).\]

It is the dominant engine of statistical inference — logistic regression, and most classifiers, are fit this way.

The connection#

Minimizing cross-entropy (like binary cross-entropy) is exactly maximizing likelihood — BCE is the negative log-likelihood of the Bernoulli model. Likelihood ratios also underlie optimal decision rules, tying estimation and decision-making together.

Theme: Probability & Statistics Foundations · All terminology

Hint

Mind map — connected ideas

Binary Cross-Entropy (BCE) · Logistic Regression · Probability Forecasts · Risk-Based Decisions · Loss Functions · Correlation

Hint

More in Probability & Statistics Foundations

Beta Distribution · Confidence Level · Correlation · Critical Value · Cumulative Distribution Function (CDF) · Frequentist · IID (Independent and Identically Distributed) · Margin of Error (MoE) · Mean · Median · Normal Distribution · Outlier · Population Proportion · Probability