CorrectiveRetrieval#

class scikitplot.corpus.CorrectiveRetrieval(plan=None, grader=None, rewrite=None, max_rounds=2, fusion=None, deadline_seconds=None, on_error='degrade', clock=None)[source]#

Grade, then accept, retry with a rewritten query, or abstain (CRAG).

Parameters:
planRetrievalExecutionPlan or None, default=None

How one round retrieves: plan.run(index, query). None is index.search(query, **search_kwargs).

gradercallable or None, default=None

grader(query, hits) -> GradeResult. None is TermCoverageGrader().

rewritecallable or None, default=None

rewrite(query, rounds) -> str or None: the next round’s query, given the original query and the rounds so far. The proposal is stripped of surrounding whitespace; None (or a rewrite returning None or a query already run) means no retry. It is called only while rounds remain and the deadline has not passed.

max_roundsint, default=2

Rounds at most, the first included: the call budget.

fusionFusionPolicy or None, default=None

Fuses the ambiguous rounds (one or more) for a partial result; None is FusionPolicy(). The leg of round i is named "r{i}". A fusion that keeps no hit (every leg weighed 0) makes the result abstained. Fusing several rounds needs each hit’s identity (doc_id); without one, the first ambiguous round is returned unfused and FUSE_FAILED is recorded. One round is cut and weighed by the policy without needing an identity.

deadline_secondsfloat or None, default=None

No rewrite is asked for and no further round starts after this many seconds (checked before the rewrite and again before the round); a stop by the deadline is recorded (DEADLINE_EXCEEDED), so the result is degraded. Without a rewrite there is no further round, and no deadline stop.

on_error{“degrade”, “raise”}, default=”degrade”

A search, grader or rewrite that raises, whatever the reason (a keyword argument the index does not take included): recorded, with the round graded incorrect ("degrade"), or propagated. A failed or cancelled round is never graded: it is not evidence.

clockcallable or None, default=None

Seconds; None is time.perf_counter.

Raises:
TypeError, ValueError

If a field is of the wrong type or out of range.

Parameters:

Examples

An index that returns only documents holding every query word finds nothing for the first query; the rewrite drops a word:

>>> from types import SimpleNamespace as NS
>>> from scikitplot.corpus import CorrectiveRetrieval, TermCoverageGrader
>>> docs = {"a": "spans are slices", "b": "the cache key is a digest"}
>>> class Index:
...     def search(self, query):
...         return [
...             NS(doc=NS(doc_id=k, text=t), score=1.0)
...             for k, t in docs.items()
...             if set(query.split()) <= set(t.split())
...         ]
>>> corrective = CorrectiveRetrieval(
...     grader=TermCoverageGrader(accept=0.6),
...     rewrite=lambda query, rounds: " ".join(query.split()[:-1]) or None,
... )
>>> result = corrective.run(Index(), "cache key size")
>>> result.verdict.value, [r.grade.grade.value for r in result.rounds]
('corrected', ['incorrect', 'correct'])
>>> [h.doc.doc_id for h in result]
['b']
bind(index, **search_kwargs)[source]#

Return query -> self.run(index, query, **search_kwargs).

Parameters:
indexobject

What every round searches.

**search_kwargs

Passed to every round.

Returns:
callable

For evaluate_retrieval.

Parameters:

  • index (Any)

  • search_kwargs (Any)

Return type:

Callable[[str], CorrectionResult]

clock: Callable[[], float] | None = None#
deadline_seconds: float | None = None#
fusion: FusionPolicy | None = None#
grader: Callable[[str, Any], GradeResult] | None = None#
max_rounds: int = 2#
on_error: str = 'degrade'#
plan: Any = None#
rewrite: Callable[[str, tuple[CorrectionRound, ...]], str | None] | None = None#
run(index, query, **search_kwargs)[source]#

Run up to max_rounds graded rounds for query.

Parameters:
indexobject

Anything with search (or what plan.run takes).

querystr

A non-empty query.

**search_kwargs

Passed to index.search (or plan.run) in every round.

Returns:
CorrectionResult
Parameters:

  • index (Any)

  • query (str)

  • search_kwargs (Any)

Return type:

CorrectionResult