🌊 Data Drift

Data Drift#

When the distribution of input data shifts away from training.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Data drift — also called covariate shift — is a change in the input distribution \(P(X)\) while the model itself stays fixed. Its weights and logic are unchanged, but the data arriving at inference no longer resembles the training data, so predictions grow less reliable.

Drift vs noise#

Data drift is systematic — a sustained, directional shift — not the random fluctuation that’s normal and expected. Examples: a fraud model meeting new devices and geographies, or a credit model trained on salaried workers now scoring gig workers.

Detecting and relating it#

It’s caught by comparing a production window to a reference window with statistical tests (KS, Chi-square), PSI, or divergence metrics — the input side, so it’s detectable before labels arrive. Data drift can evolve into concept drift, which is why teams monitor \(P(X)\) first, then investigate the input–output relationship if performance drops.

Theme: Distribution Shift & Drift · All terminology

Hint

Mind map — connected ideas

Concept Drift · Covariate Drift (a.k.a. Covariate Shift) · PSI (Population Stability Index) · KS shift (Kolmogorov–Smirnov shift) · Drift Detection · Categorical Drift

Hint

More in Distribution Shift & Drift

Cardinality in Categorical Data · Categorical Drift · Categorical Explosions · Classifier Two-Sample Tests (C2STs) · Concept Drift · Covariate Drift (a.k.a. Covariate Shift) · Dataset Shift · Drift Detection · Drift Guardrails · Energy Distance · Jensen–Shannon (JS) Divergence · KS shift (Kolmogorov–Smirnov shift) · Kullback–Leibler (KL) Divergence · Label Drift (a.k.a. Target Drift)