ScoreCalibrator#
- class scikitplot.rank_bm25.ScoreCalibrator(method='sigmoid', *, platt_targets=True)[source]#
Monotone map from BM25 scores to probabilities of relevance.
- Parameters:
- method{“sigmoid”, “isotonic”}, default=”sigmoid”
Platt scaling (two parameters, data-efficient, strictly monotone) or isotonic regression (flexible, needs more labels, can merge ranks).
- platt_targetsbool, default=True
For
"sigmoid": fit to Platt’s smoothed targets instead of 0/1, which keeps the fit finite on perfectly separable labels.
- Attributes:
- fittedbool
Whether
fithas run.- populationstr or None
The population the labels were drawn from, as declared at fit.
- prevalence_float
Share of relevant labels in the population, after any
shift_prior.- n_samples_, n_queries_int
Labelled pairs and distinct queries used to fit.
- slope_, intercept_float
Sigmoid parameters (
"sigmoid"only).- thresholds_, values_numpy.ndarray
Fitted points (
"isotonic"only).
- Parameters:
Notes
User. Fit on scores produced exactly as they will be at prediction time – same recipe, same parameters, same analyzer, same candidate selection – and evaluate only on queries not used to fit (
require_held_out). A score transformed without fitting is not a probability; this class has no unfitted transform on purpose.- fit(scores, labels, *, query_ids, population)[source]#
Fit the map on labelled (query, document) pairs.
- Parameters:
- scoresarray-like of shape (n_pairs,)
BM25 scores, finite.
- labelsarray-like of shape (n_pairs,)
Binary relevance (0/1 or
bool).- query_idsiterable of str
The query each pair belongs to; kept only as SHA-256 digests, for
require_held_out.- populationstr
What the pairs were drawn from, e.g.
"bm25-top100"; a calibrator is valid for that population only.
- Returns:
- ScoreCalibrator
self.
- Raises:
- ValueError
For mismatched lengths, non-binary labels, labels of one class only, a non-finite score, an empty
population, a sigmoid slope<= 0, or divergence on separable labels.- TypeError
For non-numeric scores or labels.
- Parameters:
- Return type:
- classmethod from_dict(state)[source]#
Rebuild a fitted calibrator from
to_dictoutput.- Raises:
- ValueError
If the schema is not this one, or a required field is missing.
- Parameters:
- Return type:
- predict_proba(scores)[source]#
Probability of relevance for each score.
- Parameters:
- scoresarray-like of shape (n,)
BM25 scores produced the way the training scores were.
- Returns:
- numpy.ndarray of float64, shape (n,)
One probability per score (one column, not scikit-learn’s two).
- Raises:
- RuntimeError
If not fitted.
- ValueError, TypeError
For non-finite, non-numeric or multi-dimensional scores.
- Parameters:
scores (Any)
- Return type:
- shift_prior(base_rate, *, population)[source]#
Return a copy valid for a population with another relevant share.
- Parameters:
- base_ratefloat
Share of relevant pairs in the new population, in
(0, 1).- populationstr
Name of the new population.
- Returns:
- ScoreCalibrator
A new calibrator;
selfis unchanged.
- Parameters:
- Return type:
Notes
User. Adds
logit(base_rate) - logit(training prevalence)to every log-odds. Exact when the scores of relevant and of non-relevant pairs are distributed the same way in both populations (only their mix differs) – true for a subsample, not for a different candidate generator. Isotonic outputs of exactly 0 or 1 stay 0 or 1.