⚖️ Predictive Parity (Calibration)

Predictive Parity (Calibration)#

A fairness criterion requiring equal positive predictive value (precision) across groups.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Predictive parity, also called calibration by group, asks that a score mean the same thing regardless of group: among everyone assigned the same predicted probability, the actual positive rate is equal across groups. Formally, for a predicted probability \(\hat{p}\),

\[P(Y=1 \mid \hat{P}=\hat{p}, A=a) = P(Y=1 \mid \hat{P}=\hat{p}, A=b) \quad \forall\, a, b,\]

with \(Y\) the true outcome, \(\hat{P}\) the score and \(A\) the protected attribute. In plain terms: if two people from different groups are both scored at 70%, then about 70% of each group should actually turn out positive.

Where it sits among fairness criteria#

This is the sufficiency criterion, \(Y \perp A \mid \hat{Y}\) — the label is independent of group once you condition on the score. It is the counterpart to the independence criterion (demographic parity) and the separation criterion (equalized odds / equal opportunity).

Example#

A loan model scores a set of applicants at 0.7 predicted default probability. If 70% of group A but only 50% of group B actually default, the score 0.7 does not carry the same meaning across groups — predictive parity is violated.

The catch: it conflicts with the others#

When the two groups have different base rates, predictive parity and equalized odds cannot both hold (except in degenerate cases). This is the fairness impossibility theorem (Kleinberg–Chouldechova): independence, separation and sufficiency are mutually incompatible whenever base rates differ, so you must choose which to prioritise.

Limitations#

A model can be perfectly calibrated yet still distribute errors unevenly — it says nothing about false-positive or false-negative rates.
Small groups look mis-calibrated from variance alone; check with enough data.

In code#

from sklearn.calibration import calibration_curve

# compare reliability curves per group
for a in groups:
    frac_pos, mean_pred = calibration_curve(y_true[A == a], y_score[A == a], n_bins=10)

Theme: Fairness & Calibration · All terminology

Hint

Mind map — connected ideas

Equalized Odds (Fairness) · Equal Opportunity (Fairness) · Demographic Parity (Statistical Parity)

Hint

More in Fairness & Calibration

Demographic Parity (Statistical Parity) · Equal Opportunity (Fairness) · Equalized Odds (Fairness) · Fairness Guardrails · Fairness parity · Four-Fifths (80%) Rule · Selection Rate