🌊 KS shift (Kolmogorov–Smirnov shift)

KS shift (Kolmogorov–Smirnov shift)#

Using the KS statistic to quantify distribution shift in a feature.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

KS shift detects data drift in a continuous feature with the two-sample Kolmogorov–Smirnov test — it compares the feature’s cumulative distribution in a reference window against the current production window and measures their largest gap:

\[D = \sup_{x}\,\big| F_{\text{ref}}(x) - F_{\text{prod}}(x) \big|.\]

A large D means the two samples likely come from different distributions.

Why it’s used#

The KS test is non-parametric — it assumes no particular distribution shape — so it flags arbitrary changes in a numeric feature’s distribution. When D exceeds a critical value (or its p-value falls below a threshold), the shift is statistically significant.

Using it in practice#

At production scale, tiny shifts become “significant” on huge samples, so the threshold is calibrated to batch size to avoid alert fatigue. KS shift complements PSI (which grades severity) and Chi-square (for categorical features) as part of a drift-monitoring suite.

Theme: Distribution Shift & Drift · All terminology

Hint

Mind map — connected ideas

Kolmogorov–Smirnov (KS) Test · Cumulative Distribution Function (CDF) · Data Drift · PSI (Population Stability Index) · Covariate Drift (a.k.a. Covariate Shift) · Concept Drift

Hint

More in Distribution Shift & Drift

Cardinality in Categorical Data · Categorical Drift · Categorical Explosions · Classifier Two-Sample Tests (C2STs) · Concept Drift · Covariate Drift (a.k.a. Covariate Shift) · Data Drift · Dataset Shift · Drift Detection · Drift Guardrails · Energy Distance · Jensen–Shannon (JS) Divergence · Kullback–Leibler (KL) Divergence · Label Drift (a.k.a. Target Drift)