Logit Space#
The pre-activation, log-odds scale on which linear models and nets operate.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
Logit space means working with the raw log-odds scores \(z\) (the logits) instead of the probabilities \(p = \sigma(z)\) — the “pre-sigmoid” world, where values span the whole real line rather than being squeezed into (0,1).
Why it’s used#
Computing the loss directly from logits is far more numerically stable. Converting a logit to a
probability and then taking its log can underflow (a tiny \(p\) rounds to 0, and \(\log 0 =
-\infty\)) or overflow (\(e^{z}\) for a large logit exceeds the float range); staying in logit space
with the log-sum-exp trick avoids both. This is why frameworks fuse sigmoid + BCE
(BCEWithLogitsLoss) and softmax + cross-entropy into a single from_logits op.
The payoff#
Better stability and cleaner gradients — the same reason log-space helps elsewhere. Logit space ties the log-odds, the sigmoid, and the cross-entropy loss into one numerically safe computation.
Theme: Model Training & Optimization · All terminology
Hint
Mind map — connected ideas
Log-Odds · Binary Cross-Entropy (BCE) · Sigmoid Function · Loss Functions · Underflow · Log-Space
Hint
More in Model Training & Optimization
Active Learning · Binary Cross-Entropy (BCE) · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logits · Loss Functions · Model Distillation (Knowledge Distillation)
See also
Source article Adapted (context, re-expressed) in our own words from: Logit Space (insightful-data-lab.com).