reliability_table#
- scikitplot.corpus.reliability_table(y_true, y_prob, *, n_bins=10, strategy='uniform')[source]#
Per-bin mean predicted probability against observed relevance frequency.
- Parameters:
- y_truearray-like of shape (n_samples,)
Binary relevance labels (0/1 or
bool).- y_probarray-like of shape (n_samples,)
Predicted probabilities of relevance, in
[0, 1].- n_binsint, default=10
Number of bins.
- strategy{“uniform”, “quantile”}, default=”uniform”
Equal-width bins over
[0, 1], or edges at quantiles ofy_prob(so bins hold similar counts).
- Returns:
- dict
edges(n_bins + 1floats), and per bincount(int),mean_predictedandobserved_frequency(NaNfor an empty bin), plusn_binsandstrategy.
- Raises:
- ValueError
For different lengths, no samples, a non-binary label, a probability outside
[0, 1]or non-finite,n_bins < 1or an unknownstrategy.- TypeError
For non-numeric input or a non-integer
n_bins.
- Parameters:
- Return type:
Notes
User. Empty bins are kept (count 0,
NaNvalues) so tables from two systems line up bin for bin; scikit-learn’scalibration_curvedrops them. Quantile edges can coincide when many probabilities are equal; such bins are empty.Examples
>>> table = reliability_table([0, 1, 1, 0], [0.1, 0.9, 0.8, 0.3], n_bins=2) >>> table["count"], table["observed_frequency"] ([2, 2], [0.0, 1.0])