MultilangConfig#

class scikitplot.corpus.MultilangConfig(*, enabled=True, include_raw_text=False, include_preprocessing_trace=False, include_semantemes=True, include_script_spans=True, include_grapheme_counts=True, include_embedding_hook=True, embedding_model_name=None, embedding_model_version=None, language_hint=None)[source]#

Multilang feature flags for any chunker.

Passed as a constructor parameter to the five chunker classes. All flags default to the lowest-cost setting so that existing code that does not set multilang_config is unaffected.

Parameters:
enabledbool

Master switch. When False the mixin skips ALL multilang processing and returns chunks with no multilang metadata key. Default True.

include_raw_textbool

Preserve the pre-normalised raw text in chunk.metadata["multilang"]["raw_text"] for comparison. Default False.

include_preprocessing_tracebool

Record every preprocessing step in chunk.metadata["multilang"]["preprocessing_trace"]. Default False.

include_semantemesbool

Build SemantemeInfo list from tokens/morphemes and store in chunk.metadata["multilang"]["semantemes"]. Default True.

include_script_spansbool

Run ScriptSegmenter on each chunk to compute per-script spans. Default True.

include_grapheme_countsbool

Compute grapheme_count and codepoint_count per chunk. Default True.

include_embedding_hookbool

Reserve the embedding key in MultilangChunkMeta for later population by an embedder. Setting this to False excludes the key entirely. Default True.

embedding_model_namestr or None

Name of the embedding model to record in script_model_version. Set this when the chunker itself is embedding-aware (e.g. SemanticChunker). Default None.

embedding_model_versionstr or None

Version tag for embedding_model_name. Default None.

language_hintstr or None

BCP-47 language code override. When set, overrides auto-detection for all chunks produced in this run. Default None.

Parameters:

  • enabled (bool)

  • include_raw_text (bool)

  • include_preprocessing_trace (bool)

  • include_semantemes (bool)

  • include_script_spans (bool)

  • include_grapheme_counts (bool)

  • include_embedding_hook (bool)

  • embedding_model_name (str | None)

  • embedding_model_version (str | None)

  • language_hint (str | None)

Examples

>>> cfg = MultilangConfig(
...     include_raw_text=True,
...     include_preprocessing_trace=True,
...     include_semantemes=True,
... )
>>> cfg.enabled
True