redact_documents#
- scikitplot.cleanprompt.redact_documents(documents, cleaner, format='text')[source]#
Return corpus documents with every text field redacted by one cleaner.
- Parameters:
- documentsiterable of CorpusDocument
Documents from any corpus reader or pipeline.
- cleanerCleaner
The cleaner whose vault collects what is removed; decode replies with it.
- formatstr, default=’text’
The format each text field is read as. It must be selected by the cleaner’s plan.
- Returns:
- list of CorpusDocument
New documents, in the same order. The inputs are unchanged. Each
doc_idstays in its own scheme: the content-addressed SHA-256 of the redacted text, or, for a document named bydoc_id_scheme="uuid7", the same UUIDv7 when the text is unchanged and a newly minted one when it is not. Parent links within the batch follow.
- Raises:
- CleanPromptError
If any field cannot be redacted, or a
parent_doc_idnames two documents of the batch that were renamed differently. Nothing is returned partly redacted.
- Parameters:
- Return type:
Examples
>>> from scikitplot.corpus._schema import CorpusDocument >>> from scikitplot.cleanprompt._plan import FluentCleanPrompt >>> doc = CorpusDocument.create( ... "n.txt", 0, "Mail ann@example.com" ... ) >>> redact_documents([doc], FluentCleanPrompt().materialize())[ ... 0 ... ].text 'Mail [EMAIL-1]'
Gallery examples#
Cleanprompt Packs and Formats: Records, Configs and Whole Folders