Binary Cross-Entropy (BCE)#
The standard loss for binary classification, penalising confident wrong predictions.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
Binary cross-entropy (log loss) is the standard loss for binary classification, for a true label \(y \in \{0,1\}\) and predicted probability \(p = \sigma(z)\):
averaged over the data.
How it behaves#
The logarithm gives an asymmetric penalty — a confident-correct prediction costs almost nothing (\(-\log 0.99 \approx 0.01\)), a confident-wrong one costs a lot (\(-\log 0.01 \approx 4.6\)). This pressures the model to be confident when right and hesitant when wrong, pushing toward calibrated probabilities.
Why it fits#
BCE is the negative log-likelihood of the Bernoulli distribution, so minimizing it is maximum likelihood — the natural partner of the sigmoid, with a clean gradient \((p - y)\). Its multi-class analogue is categorical cross-entropy with softmax.
Theme: Model Training & Optimization · All terminology
Hint
Mind map — connected ideas
Loss Functions · Logit Space · Sigmoid Function · Log-Odds · Binary Classification · Classification Probability
Hint
More in Model Training & Optimization
Active Learning · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Logits · Loss Functions · Model Distillation (Knowledge Distillation)
See also
Source article Adapted (context, re-expressed) in our own words from: Binary Cross-Entropy (BCE) (insightful-data-lab.com).