Active Learning#
Iteratively querying the most informative examples to label, cutting labelling cost.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
Active learning trains a model iteratively and lets it choose the most informative examples to label, instead of labelling everything. The goal is high accuracy from far fewer labels — invaluable when annotation is expensive or slow, as with medical images or legal documents.
The loop#
Start with a small labelled set and a large unlabelled pool. Train an initial model, use a query strategy to pick the most valuable unlabelled samples, send them to an oracle (a human expert) for labels, add them to the training set and retrain — repeating until the model is good enough or the labelling budget runs out.
Query strategies#
How to choose what to label. Uncertainty sampling picks the least confident cases (for binary classification, predicted probability near 0.5). Query by committee trains several models and picks where they disagree most. Expected model change chooses points that would most move the model. Diversity sampling picks examples unlike the existing training data to cover the input space.
Why it works#
Given 100,000 unlabelled emails at $2 each to label, training on 1,000 and then querying the
500 most uncertain improves accuracy faster than random labelling. Active learning cuts
annotation cost, accelerates learning, and naturally prioritises rare or uncertain cases —
helping with imbalance for free.
Theme: Model Training & Optimization · All terminology
Hint
Mind map — connected ideas
Reweighting · Continuous Retraining · Ensemble · Medical AI · Monitoring Pipelines · Bayesian Correction
Hint
More in Model Training & Optimization
Binary Cross-Entropy (BCE) · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Logits · Loss Functions · Model Distillation (Knowledge Distillation)
See also
Source article Adapted (context, re-expressed) in our own words from: Active Learning (insightful-data-lab.com).