reliability_table#

scikitplot.corpus.reliability_table(y_true, y_prob, *, n_bins=10, strategy='uniform')[source]#

Per-bin mean predicted probability against observed relevance frequency.

Parameters:
y_truearray-like of shape (n_samples,)

Binary relevance labels (0/1 or bool).

y_probarray-like of shape (n_samples,)

Predicted probabilities of relevance, in [0, 1].

n_binsint, default=10

Number of bins.

strategy{“uniform”, “quantile”}, default=”uniform”

Equal-width bins over [0, 1], or edges at quantiles of y_prob (so bins hold similar counts).

Returns:
dict

edges (n_bins + 1 floats), and per bin count (int), mean_predicted and observed_frequency (NaN for an empty bin), plus n_bins and strategy.

Raises:
ValueError

For different lengths, no samples, a non-binary label, a probability outside [0, 1] or non-finite, n_bins < 1 or an unknown strategy.

TypeError

For non-numeric input or a non-integer n_bins.

Parameters:
Return type:

dict[str, Any]

Notes

User. Empty bins are kept (count 0, NaN values) so tables from two systems line up bin for bin; scikit-learn’s calibration_curve drops them. Quantile edges can coincide when many probabilities are equal; such bins are empty.

Examples

>>> table = reliability_table([0, 1, 1, 0], [0.1, 0.9, 0.8, 0.3], n_bins=2)
>>> table["count"], table["observed_frequency"]
([2, 2], [0.0, 1.0])