Nominal Classification Models: Model State and Evaluation Metrics#

Stage 8 · 📊 Model Evaluation · Lesson 52 of 56 · advanced

◀ Previous · Binary Classification Models – Conceptual Framework and Evaluation Metrics · Next · Binary Classification Model Evaluation and Threshold Optimization ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Beyond two classes#

Many classification problems have more than two unordered labels — payment type (cash / card / mobile), a product category, a species. These are nominal (multi-class) problems: the classes have no natural order, and every prediction must pick one of \(K\) labels. Evaluation carries over from the binary case, but with two twists worth understanding.

The K×K confusion matrix#

The confusion matrix generalises directly: it becomes a \(K \times K\) table, actual classes down the rows and predicted classes across the columns. The diagonal still holds the correct predictions, and every off-diagonal cell now names a specific confusion — how often class A was mistaken for class B. This is the matrix’s great virtue in multi-class work: it shows which pairs of classes the model muddles (cash vs mobile, say), pointing directly at what to fix. scikit-plots renders it as the same labelled heat-map plot, just larger.

Per-class metrics#

Precision, recall and F1 are binary notions, so for \(K\) classes they are computed per class, one-vs-rest: for class A, treat A as “positive” and everything else as “negative”, and read TP, FP and FN from the matrix’s row and column for A. Each class gets its own precision (of the predictions of A, how many were right?) and recall (of the true A’s, how many were found?) — scikit-learn’s classification_report prints exactly this table, one row per class.

Averaging: macro, micro, weighted#

To summarise the per-class scores in one number, three averages are standard, and they answer different questions:

Macro — the plain mean of the per-class scores: every class counts equally, however rare, so it exposes weakness on small classes;
Weighted — the mean weighted by class frequency: large classes dominate, mirroring overall behaviour;
Micro — pool all the TP / FP / FN counts first, then compute: every instance counts equally. For single-label problems micro-precision, micro-recall and accuracy all coincide.

On imbalanced data the choice matters: a model useless on a rare class can still post a high weighted or micro score, while its macro score collapses — so pick the average that matches whether classes or instances are what the business weighs equally.

Hint