Reliability Curves (also called Calibration Curves)#
Plots of predicted probability against observed frequency.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
A reliability curve (or calibration curve / reliability diagram) is the visual check for calibration — it plots predicted probability on the x-axis against the observed frequency of the outcome on the y-axis. A perfectly calibrated model traces the diagonal \(y = x\).
Reading it#
Points below the diagonal mean the model is overconfident (accuracy falls short of its confidence); points above mean it is underconfident. A companion histogram of confidences shows whether predictions pile up at the extremes — a hallmark of overconfident networks.
Why it complements ECE#
A single ECE number can’t say where miscalibration happens, and two models with the same ECE can have very different curves. The reliability curve localizes the problem across the confidence range — and the weighted gap between it and the diagonal is the ECE.
Theme: Probability Calibration · All terminology
Hint
Mind map — connected ideas
Expected Calibration Error (ECE) · Overconfident · Underconfident · Confidence Level · Brier Score · Temperature Scaling
Hint
More in Probability Calibration
Adaptive ECE (Expected Calibration Error with Adaptive Binning) · Brier Score · Calibration quality (Model Calibration) · Expected Calibration Error (ECE) · Isotonic Regression · Maximum Calibration Error (MCE) · Murphy’s Decomposition · Overconfident · Platt Scaling · Temperature Scaling · Underconfident
See also
Source article Adapted (context, re-expressed) in our own words from: Reliability Curves (also called Calibration Curves) (insightful-data-lab.com).