Maximum Calibration Error (MCE)#
The largest gap between confidence and accuracy across calibration bins.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
Maximum Calibration Error reports the worst calibration gap rather than the average — the largest difference between accuracy and confidence over all bins:
Where ECE asks how miscalibrated on average?, MCE asks how bad does it get?
When it matters#
MCE is the right lens for safety-critical systems — medical, autonomous, financial — where a single badly miscalibrated confidence region can cause harm, even if the average looks fine. Lower is better, as with ECE.
Its limits#
Like ECE it is binning-dependent (the answer shifts with bin count and scheme), and it is not a proper scoring rule — a model can achieve low calibration error with trivial predictions, so MCE must be read alongside discrimination metrics, not alone.
Theme: Probability Calibration · All terminology
Hint
Mind map — connected ideas
Adaptive ECE (Expected Calibration Error with Adaptive Binning) · Murphy’s Decomposition · Confidence Level · Temperature Scaling · Risk-Based Decisions · Overconfident
Hint
More in Probability Calibration
Adaptive ECE (Expected Calibration Error with Adaptive Binning) · Brier Score · Calibration quality (Model Calibration) · Expected Calibration Error (ECE) · Isotonic Regression · Murphy’s Decomposition · Overconfident · Platt Scaling · Reliability Curves (also called Calibration Curves) · Temperature Scaling · Underconfident
See also
Source article Adapted (context, re-expressed) in our own words from: Maximum Calibration Error (MCE) (insightful-data-lab.com).