Squashing Function#
Any bounded nonlinearity (sigmoid, tanh) that compresses its input range.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
A squashing function is any function that compresses an unbounded input — any real number in \((-\infty, \infty)\) — into a bounded range, giving the characteristic S-shape. It “squashes” an infinite domain into a finite interval.
Examples#
The sigmoid squashes to (0, 1) (a probability), tanh to (−1, 1), and softmax squashes a vector of logits into probabilities on (0,1). They are the classic non-linear activations that let a network turn raw scores into interpretable, constrained outputs.
Why it matters#
Squashing is what converts an unbounded linear score (like the log-odds) into something usable — a probability, or a normalized signal — and the non-linearity is what lets stacked layers model complex patterns. Its flat tails are also the source of saturation and vanishing gradients.
Theme: Model Training & Optimization · All terminology
Hint
Mind map — connected ideas
Sigmoid Function · Softmax Function · Neural Networks · Log-Odds · Binary Classification · Classification Probability
Hint
More in Model Training & Optimization
Active Learning · Binary Cross-Entropy (BCE) · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Logits · Loss Functions
See also
Source article Adapted (context, re-expressed) in our own words from: Squashing Function (insightful-data-lab.com).