NerDetector#

class scikitplot.cleanprompt.NerDetector(model=None, labels=frozenset({'EVENT', 'FAC', 'GPE', 'LANGUAGE', 'LAW', 'LOC', 'NORP', 'ORG', 'PERSON', 'PRODUCT', 'WORK_OF_ART'}), kind_prefix='', priority=None, confidence=None, disable=None, language='en', size='sm')[source]#

Detect named entities with a spaCy pipeline.

Parameters:
modelstr, default=’en_core_web_lg’

Name of an installed spaCy model.

labelsiterable of str, optional

Entity labels to redact. None redacts every label the model reports. Defaults to DEFAULT_ENTITY_LABELS.

kind_prefixstr, default=’’

Prepended to the spaCy label to form the placeholder category, so kind_prefix="NE_" yields [NE_PERSON-1].

priorityint, default=30

Arbitration weight. Below the structural patterns by default: an email address recognised by both should be labelled EMAIL, not ORG.

confidencefloat, default=0.7

Reported confidence. spaCy’s entity recogniser does not expose a calibrated per-entity score, so this is a fixed, documented value used for filtering and reporting, never presented as a probability.

disableiterable of str, optional

Pipeline components to disable at load time. Defaults to disabling lemmatizer and textcat, which the entity recogniser does not need.

Raises:
CapabilityError

On first detection, if the ner tier is unavailable or the model is not installed.

Parameters:

  • model (str | None)

  • labels (Iterable[str] | None)

  • kind_prefix (str)

  • priority (int | None)

  • confidence (float | None)

  • disable (Iterable[str] | None)

  • language (str)

  • size (str)

Notes

Developer notes. sentencizer-style components are left alone; only components irrelevant to entity recognition are disabled, because disabling tok2vec or ner itself would silently produce zero entities, which in a privacy tool reads as “nothing sensitive found”.

Examples

>>> detector = NerDetector()
>>> detector.kind
'NE'
confidence#

!! processed by numpydoc !!

detect(text, policy)[source]#

Yield one span per accepted named entity.

Parameters:
textstr

The original text.

policyRedactionPolicy

Supplies Limits.max_input_chars, which is propagated to the pipeline’s own document ceiling.

Yields:
Span

Entities whose label is selected, in ascending start order.

Raises:
CapabilityError

If the tier or the model is unavailable.

DetectorError

If the pipeline raises while processing the text.

Parameters:
Return type:

Iterator[Span]

Notes

Developer notes. spaCy’s max_length guard exists to stop a very long document exhausting memory during parsing. It is aligned with this submodule’s own bound so that one limit governs, and a document that the base tier accepts is not rejected by the NER tier at a different size.

Entities reported by spaCy never overlap, so the spans yielded here are already disjoint among themselves; they may still overlap structural detections, which is what resolve_spans settles.

kind#

!! processed by numpydoc !!

kind_prefix#

!! processed by numpydoc !!

kinds()[source]#

Return the placeholder categories this detector can produce.

Returns:
tuple of str

Sorted category names, or an empty tuple when every label is accepted and the set is therefore open.

Return type:

tuple[str, …]

Notes

Developer notes. A registry needs this because Detector.kind is a single value, whereas this detector produces one category per entity label. detect sets each span’s kind from the entity label, and the registry selects this detector by its declared kind of "NE".

labels: frozenset[str] | None#

!! processed by numpydoc !!

language#

!! processed by numpydoc !!

model#

!! processed by numpydoc !!

name#

!! processed by numpydoc !!

priority#

!! processed by numpydoc !!

reads_view = False#

Whether detect is a pure function of the text it is given, so that it may also read the detection view (CP-098) and have its spans mapped back onto the original. False by default: a detector bound to one document — offsets computed in advance from the original, such as a field or region detector — would have its spans mapped twice and cut through a file’s structure (CP-102). Opt in only when every offset a detector yields indexes the string passed to detect.

resolution#

!! processed by numpydoc !!