🏋️ Logits

Logits#

Raw pre-activation scores before a sigmoid or softmax.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Logits are the raw, unnormalized scores a classifier’s final layer produces before they’re turned into probabilities. They range over all real numbers — positive, negative, unbounded — and live in log-odds space, not probability space.

From logits to probabilities#

A softmax turns a vector of logits into a probability distribution that sums to 1 (for multiclass), while a sigmoid maps a single logit to one probability (for binary). Because softmax normalizes, raising one logit lowers the others’ probabilities — the competition that sharpens a prediction.

Why keep them raw#

Exposing logits enables numerically stable training (log-softmax beats probabilities-then-log, which is why frameworks feed cross-entropy raw logits) and post-hoc calibration — temperature scaling divides logits by T before softmax, which is only possible when the logits are available.

Theme: Model Training & Optimization · All terminology

Hint

Mind map — connected ideas

Softmax Function · Sigmoid Function · Logit Space · Log-Odds · Temperature Scaling · Log Loss (also called Logarithmic Loss or Cross-Entropy Loss)

Hint

More in Model Training & Optimization

Active Learning · Binary Cross-Entropy (BCE) · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Loss Functions · Model Distillation (Knowledge Distillation)