reserved_label_spans#

scikitplot.cleanprompt.reserved_label_spans(text, policy)[source]#

Find substrings of text that already match the placeholder grammar.

Parameters:
textstr

The original text.

policyRedactionPolicy

Supplies the placeholder grammar.

Returns:
rangestuple of tuple of int

(start, end) pairs to exclude from detection, ascending.

labelsset of str

The label strings occupying those ranges, excluded from allocation.

Parameters:
Return type:

tuple[tuple[tuple[int, int], …], set[str]]

Notes

Developer notes. Both halves matter and they solve different problems. Excluding the ranges gives idempotence: a detector cannot tag the inside of a placeholder. Excluding the labels prevents an aliasing bug: for the input "[EMAIL-1] and bob@example.com" the real address would otherwise be issued [EMAIL-1] as well, and restoration would then rewrite both occurrences to the same value.