resolve_spans#

scikitplot.cleanprompt.resolve_spans(spans, policy=RedactionPolicy(kinds=None, allow=(), tag_style=TagStyle(prefix='[', suffix=']', separator='-', uppercase_kind=True, style='placeholder', surrogates=None), overlap=<OverlapStrategy.LONGEST_WINS: 'LONGEST_WINS'>, limits=Limits(max_input_chars=1000000, max_spans=100000, max_entries=50000, max_literal_terms=10000, max_literal_length=4096), case_insensitive=False, min_confidence=0.0, preserve_placeholders=True), text=None)[source]#

Reduce possibly overlapping spans to a disjoint, ascending sequence.

Parameters:
spansiterable of Span

Detections over one text. May overlap in any pattern.

policyRedactionPolicy, default=DEFAULT_POLICY

Supplies overlap.

textstr, optional

The source text the spans index into. Required only when a cluster has to be merged, because the merged span’s text must be the real surface rather than a member’s.

Returns:
tuple of Span

Pairwise disjoint spans in ascending start order. Every returned span’s text equals text[start:end] when text was supplied.

Raises:
OverlapError

If two spans overlap and the strategy is STRICT.

PolicyError

If a cluster must be merged but text was not supplied.

Parameters:
Return type:

tuple[Span, …]

Notes

Developer notes. The guarantee is coverage preservation: every character covered by any input span is covered by exactly one output span. That is what makes the redaction safe under partial overlap, and it is checked directly by test__engine.py::test_resolution_preserves_coverage.

Identical duplicate spans from two detectors collapse into one cluster and therefore into one output span, so a value matched by both a pattern and a literal term is replaced once.

Examples

>>> a = Span(0, 10, "URL", "x" * 10, "regex:URL")
>>> b = Span(4, 8, "EMAIL", "x" * 4, "regex:EMAIL")
>>> [(s.start, s.end, s.kind) for s in resolve_spans([a, b])]
[(0, 10, 'URL')]