🌊 Jensen–Shannon (JS) Divergence

Jensen–Shannon (JS) Divergence#

A symmetric, bounded measure of difference between two distributions.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

JS divergence is the symmetric, bounded repair of KL. It averages the KL of each distribution to their mixture \(M = \tfrac{1}{2}(P+Q)\):

\[D_{\text{JS}}(P \,\|\, Q) = \tfrac{1}{2}D_{\text{KL}}(P \,\|\, M) + \tfrac{1}{2}D_{\text{KL}}(Q \,\|\, M).\]

Unlike KL it is always finite and symmetric.

Its properties#

JS ranges from 0 (identical) to a bounded maximum (1 in bits, \(\log 2\) in nats, when the distributions are disjoint). Its square root is the Jensen–Shannon distance, which is a proper metric — so JS gives a well-behaved, comparable measure of distributional difference.

Where it’s used#

It works on numerical and categorical features alike and is a popular drift signal — stable, less noisy, and slightly more sensitive than KL or PSI — which is why monitoring systems favour it when a symmetric, bounded score is wanted.

Theme: Distribution Shift & Drift · All terminology

Hint

Mind map — connected ideas

Kullback–Leibler (KL) Divergence · Kolmogorov–Smirnov (KS) Test · Data Drift · Concept Drift · Statistical Tests · Cramér’s V

Hint

More in Distribution Shift & Drift

Cardinality in Categorical Data · Categorical Drift · Categorical Explosions · Classifier Two-Sample Tests (C2STs) · Concept Drift · Covariate Drift (a.k.a. Covariate Shift) · Data Drift · Dataset Shift · Drift Detection · Drift Guardrails · Energy Distance · KS shift (Kolmogorov–Smirnov shift) · Kullback–Leibler (KL) Divergence · Label Drift (a.k.a. Target Drift)