🏋️ Early Stopping

Early Stopping#

Halting training when validation performance stops improving, to curb overfitting.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Early stopping is a regularisation technique: stop training before the model overfits. Rather than running a fixed number of epochs, you watch a validation metric and halt once it stops improving — capturing the model at its generalisation “sweet spot” and saving the wasted epochs beyond it.

How it works#

Hold out a validation set, train across epochs, and after each one score the validation metric and remember the best so far. If it fails to improve for N consecutive epochs — the patience — stop. The key parameters are the monitored metric, the mode (min for loss, max for accuracy or AUC), the patience, and restore-best-weights, which rolls the model back to its best epoch.

Example#

Capped at 100 epochs, suppose validation loss improves until epoch 25 and then climbs as the model starts overfitting. With patience = 3, training stops at epoch 28 and restores the weights from epoch 25 — the genuine best.

Benefits and drawbacks#

It prevents overfitting, cuts training time, and finds a near-optimal stopping point automatically. The costs are minor: it needs a validation set, and a noisy metric can trip it too soon — which is exactly what patience is there to absorb.

Theme: Model Training & Optimization · All terminology

Hint

Mind map — connected ideas

Epochs · Cross-Validation (CV) · Model Distillation (Knowledge Distillation) · Frozen Encoder · Autoencoder · Quantization

Hint

More in Model Training & Optimization

Active Learning · Binary Cross-Entropy (BCE) · Deep Ensembles · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Logits · Loss Functions · Model Distillation (Knowledge Distillation)