🔬 KS Statistic (Kolmogorov–Smirnov Statistic)

KS Statistic (Kolmogorov–Smirnov Statistic)#

The maximum gap between two cumulative distributions; measures separation or shift.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

The KS statistic measures the maximum difference between two cumulative distribution functions (CDFs). In statistics it underpins the KS test — do two samples come from the same distribution? In ML, especially credit scoring and binary classification, it measures how well a model separates positives from negatives: far-apart distributions give a large KS, heavily overlapping ones a small KS.

The formula#

For a binary problem, build the CDF of positives and the CDF of negatives across the score, then take the largest vertical gap between them:

\[KS = \max_x \left| F_{\text{pos}}(x) - F_{\text{neg}}(x) \right|,\]

where \(F\) is a cumulative distribution. Because it scans every threshold for the single point of greatest separation, KS is threshold-independent.

A worked example#

Score 1,000 customers — 500 good, 500 bad — and sort by predicted score. At each threshold track the share of bads captured against the share of goods captured; KS is the largest gap between those curves. If at score 0.65 the CDF of bads is 0.70 and the CDF of goods is 0.30, the gap is 0.40, so KS = 40%.

Reading it, and its cousin AUC#

KS runs from 0 to 1: 0 means no separating power (identical distributions), 0.4-0.6 is strong (typical in credit risk), below 0.2 is weak. It is closely related to AUC — both measure class separability and both are threshold-independent — but where AUC integrates the whole ROC curve, KS reports only the single maximum point of separation.

Theme: Model Evaluation & Uncertainty · All terminology

Hint

Mind map — connected ideas

Discriminatory Power · Gini Coefficient · Multiclass AUROC · Energy Distance · PSI (Population Stability Index)

Hint

More in Model Evaluation & Uncertainty

Average Absolute Error (AAE) · Baseline Heuristics · Bootstrap · Bootstrap Confidence Intervals (CIs) · Coverage · Cramér’s V · DeLong’s Test · Likelihood Ratio (LR) · Mann–Whitney U Test (also called the Wilcoxon rank-sum test) · MASE (Mean Absolute Scaled Error) · Mean Absolute Error (MAE) · Mean Absolute Percentage Error (MAPE) · Mean Squared Error (MSE) · Relative accuracy