📏 Multiclass AUROC

Multiclass AUROC#

AUROC extended beyond two classes via One-vs-Rest or One-vs-One schemes.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

AUROC — the probability a random positive outscores a random negative, from 0.5 (chance) to 1.0 (perfect) — is defined only for a binary problem. Multiclass AUROC is the umbrella for the strategies that extend it to \(K\) classes by reducing the problem to many binary ones and aggregating.

Two reductions#

one-vs-rest (OvR)#

For each class, score it against all others combined — \(K\) binary AUROCs:

\[\text{AUROC}_{\text{macro}} = \frac{1}{K} \sum_{i=1}^{K} \text{AUROC}(\text{class}_i \;\text{vs}\; \text{rest}).\]

one-vs-one (OvO)#

Score every pair of classes and average over all \(\binom{K}{2}\) pairs:

\[\text{AUROC}_{\text{ovo}} = \frac{2}{K(K-1)} \sum_{i<j} \text{AUROC}(\text{class}_i \;\text{vs}\; \text{class}_j).\]

OvO gives a more balanced picture under heavy imbalance, at \(O(K^2)\) cost.

Two averagings#

Independently of OvR / OvO, the per-binary results can be combined as macro (equal weight per class — fair to minorities), micro (pool all decisions into one global AUROC — sample-weighted, majority-driven), or weighted (by class frequency).

Worked example#

OvR AUROCs of 0.82, 0.75, 0.70 over three classes:

\[\text{AUROC}_{\text{macro}} = \frac{0.82 + 0.75 + 0.70}{3} = 0.7567.\]

Interpretation#

All variants keep the binary AUROC scale (0.5 random, 1.0 perfect). Read macro for fairness across classes, micro for overall sample-level discrimination, and OvO for pairwise separability. Reporting more than one avoids being misled by a single summary.

In code#

from sklearn.metrics import roc_auc_score

ovr_macro = roc_auc_score(y_true, y_score, multi_class="ovr", average="macro")
ovo_macro = roc_auc_score(y_true, y_score, multi_class="ovo", average="macro")

Theme: Classification & Averaging Metrics · All terminology

Hint

Mind map — connected ideas

One-vs-Rest (OvR) AUROC · Micro AUROC · Macro AUROC (Macro-Averaged AUROC) · Gini Coefficient

Hint

More in Classification & Averaging Metrics

Accuracy · AUC (Area Under the Curve) · Average Precision (AP) · Binary Classification · Classification Probability · Discriminatory Power · F1-score · Gini Coefficient · Harmonic Mean · Log Loss (also called Logarithmic Loss or Cross-Entropy Loss) · Macro AUC · Macro AUROC (Macro-Averaged AUROC) · Macro Averaging · Macro F1