🌊 Cardinality in Categorical Data

Cardinality in Categorical Data#

The number of distinct values a categorical feature can take.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Cardinality is the number of unique categories in a categorical feature. Gender has just two values — low cardinality; a Zip Code field has thousands — high cardinality. The distinction matters because it changes how features should be measured and encoded.

Why it matters for association#

Cardinality directly affects measures like Cramér’s V, which divides by the smaller table dimension \(k = \min(\text{rows}, \text{cols})\). With low cardinality (Gender, Yes/No) the statistic is easy to read — Gender versus product preference giving V = 0.3 is a clear moderate association. With high cardinality (zip codes, product IDs) it grows unreliable: many categories have tiny counts, the chi-square statistic inflates, and an association can look strong when it is really just sparsity.

Handling high cardinality#

Three remedies. Group rare categories into an “Other” bucket. Use target encoding or frequency encoding rather than raw category comparison. And if using Cramér’s V, ensure the sample is large enough that expected cell counts are not tiny.

An example#

Comparing City (100 categories) against Purchase (Yes/No) might yield Cramér’s V of 0.6, suggesting a strong link — but that can simply reflect too few samples per city, not a real effect of city on purchasing.

Theme: Distribution Shift & Drift · All terminology

Hint

Mind map — connected ideas

Categorical Drift · Cramér’s V · Embedding · Data Drift · Drift Detection · PSI (Population Stability Index)

Hint

More in Distribution Shift & Drift

Categorical Drift · Categorical Explosions · Classifier Two-Sample Tests (C2STs) · Concept Drift · Covariate Drift (a.k.a. Covariate Shift) · Data Drift · Dataset Shift · Drift Detection · Drift Guardrails · Energy Distance · Jensen–Shannon (JS) Divergence · KS shift (Kolmogorov–Smirnov shift) · Kullback–Leibler (KL) Divergence · Label Drift (a.k.a. Target Drift)