expected_calibration_error#

scikitplot.corpus.expected_calibration_error(y_true, y_prob, *, n_bins=10, strategy='uniform')[source]#

Count-weighted mean absolute gap between prediction and observation.

Parameters:
y_truearray-like of shape (n_samples,)

Binary relevance labels.

y_probarray-like of shape (n_samples,)

Predicted probabilities of relevance.

n_binsint, default=10

Number of bins.

strategy{“uniform”, “quantile”}, default=”uniform”

See reliability_table.

Returns:
float

sum_b (n_b / N) * |observed_b - mean_predicted_b| over non-empty bins; in [0, 1], 0 is perfectly calibrated.

Raises:
ValueError, TypeError

See reliability_table.

Parameters:
Return type:

float

Examples

>>> round(
...     expected_calibration_error([0, 1, 1, 0], [0.1, 0.9, 0.8, 0.3], n_bins=2), 4
... )
0.175