📏 Multi-label Classification

Multi-label Classification#

Tasks where each instance may carry several non-exclusive labels at once.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

In multi-label classification each sample can carry several labels at once. This is different from multi-class classification, where every sample gets exactly one label from a set. Here the labels are not mutually exclusive — the model makes an independent yes/no decision for every class:

\[f : X \;\rightarrow\; \{0, 1\}^K,\]

so for \(K\) classes the output is a length-\(K\) vector of binary decisions. A movie can be Action + Comedy, a news story Politics + Economy, a patient diabetic + hypertensive, an image dog + car + tree.

Multi-label vs multi-class#

Feature	Multi-class	Multi-label
Labels per sample	Exactly one	One or more
Class exclusivity	Mutually exclusive	Independent
Output layer	Softmax (probabilities sum to 1)	Sigmoid (independent probability per class)
Loss	Categorical cross-entropy	Binary cross-entropy per class

How it’s modelled#

The network ends in a sigmoid per class rather than a single softmax, and each output is thresholded independently. The loss is binary cross-entropy summed or averaged over the \(K\) labels — effectively \(K\) coupled binary problems sharing one backbone.

How it’s scored#

Because a prediction is a set of labels, the metrics differ from single-label ones:

Per-label precision / recall / F1, then aggregated with micro, macro or weighted averaging.
Hamming loss — the fraction of individual label slots that are wrong.
Subset accuracy — strict: 1 only if every label of the sample is correct, else 0.
Jaccard similarity — intersection over union of predicted and true label sets.

Worked example#

True labels for an image: \(\{\text{Cat}, \text{Dog}\}\); the model predicts \(\{\text{Cat}, \text{Horse}\}\).

Precision \(= 1/(1+1) = 0.5\) (Cat right, Horse is a false positive).
Recall \(= 1/(1+1) = 0.5\) (Dog was missed — a false negative).
Jaccard \(= |\{\text{Cat}\}| / |\{\text{Cat},\text{Dog},\text{Horse}\}| = 1/3 \approx 0.33\).

Pitfalls and edge cases#

Subset accuracy is harsh — one wrong label out of many zeroes the whole sample; report it alongside Hamming loss, not alone.
Per-label thresholds — a single 0.5 cutoff is rarely optimal for every label; tune thresholds per class, especially under imbalance.
Label imbalance and correlation — rare labels and co-occurring labels (Cat with Dog) are easy to under-predict; micro averaging will hide that, macro will expose it.

Theme: Classification & Averaging Metrics · All terminology

Hint

Mind map — connected ideas

Single-label Classification · Micro F1 · Macro F1 · Micro AUROC · Macro AUROC (Macro-Averaged AUROC)

Hint

More in Classification & Averaging Metrics

Accuracy · AUC (Area Under the Curve) · Average Precision (AP) · Binary Classification · Classification Probability · Discriminatory Power · F1-score · Gini Coefficient · Harmonic Mean · Log Loss (also called Logarithmic Loss or Cross-Entropy Loss) · Macro AUC · Macro AUROC (Macro-Averaged AUROC) · Macro Averaging · Macro F1