redact_documents#

scikitplot.cleanprompt.redact_documents(documents, cleaner, format='text')[source]#

Return corpus documents with every text field redacted by one cleaner.

Parameters:
documentsiterable of CorpusDocument

Documents from any corpus reader or pipeline.

cleanerCleaner

The cleaner whose vault collects what is removed; decode replies with it.

formatstr, default=’text’

The format each text field is read as. It must be selected by the cleaner’s plan.

Returns:
list of CorpusDocument

New documents, in the same order. The inputs are unchanged. Each doc_id stays in its own scheme: the content-addressed SHA-256 of the redacted text, or, for a document named by doc_id_scheme="uuid7", the same UUIDv7 when the text is unchanged and a newly minted one when it is not. Parent links within the batch follow.

Raises:
CleanPromptError

If any field cannot be redacted, or a parent_doc_id names two documents of the batch that were renamed differently. Nothing is returned partly redacted.

Parameters:
Return type:

list[Any]

Examples

>>> from scikitplot.corpus._schema import CorpusDocument
>>> from scikitplot.cleanprompt._plan import FluentCleanPrompt
>>> doc = CorpusDocument.create(
...     "n.txt", 0, "Mail ann@example.com"
... )
>>> redact_documents([doc], FluentCleanPrompt().materialize())[
...     0
... ].text
'Mail [EMAIL-1]'