PreprocessingStep#

class scikitplot.corpus.PreprocessingStep(name, description, changed, char_delta, grapheme_delta=None, params=None, input_hash=None, output_hash=None, duration_s=None)[source]#

Record of a single preprocessing transformation applied to raw text.

Used to build a PreprocessingTrace that makes the preprocessing pipeline fully retrospective — every change is documented with enough information to verify, replay, or invert it.

Parameters:
namestr

Short identifier of the transformation (e.g. "nfc_normalise", "strip_control", "bom_strip", "whitespace_collapse", "ocr_correction").

descriptionstr

Human-readable explanation of what this step does.

changedbool

True if this step actually modified the text. False when the text was already in the expected form (idempotent execution).

char_deltaint

Change in codepoint count: len(after) - len(before). Negative for stripping steps; zero for reordering steps.

grapheme_deltaint or None

Change in grapheme cluster count. None if grapheme counting was not available at this step (regex not installed).

paramsMetadataDict or None

Configuration parameters used for this step. Must be JSON-serialisable. None if the step has no parameters.

input_hashstr or None

MD5 hex of the text before this step. Allows exact change detection without storing the full text. None if not tracked.

output_hashstr or None

MD5 hex of the text after this step.

Parameters:

  • name (str)

  • description (str)

  • changed (bool)

  • char_delta (int)

  • grapheme_delta (int | None)

  • params (dict[str, Any] | None)

  • input_hash (str | None)

  • output_hash (str | None)

  • duration_s (float | None)

Notes

Developer note: Hash all PreprocessingStep objects into a stable fingerprint to build a preprocessing fingerprint for idempotency verification on pipeline re-runs. The fingerprint is: hashlib.md5(",".join(s.name for s in trace.steps).encode()).hexdigest()

changed: bool[source]#
char_delta: int[source]#
description: str[source]#
duration_s: float | None = None#

Wall-clock seconds this step took. None when not timed.

grapheme_delta: int | None = None#
input_hash: str | None = None#
name: str[source]#
output_hash: str | None = None#
params: dict[str, Any] | None = None#
to_dict()[source]#

Serialise to a JSON-safe dict.

Returns:
MetadataDict
Return type:

dict[str, Any]