WritingSystemAdapter#

class scikitplot.corpus.WritingSystemAdapter(config=None, *, extra_strategies=None)[source]#

Dispatch per-script segmentation to the correct strategy.

Receives a list of ScriptSpan objects (produced by ScriptSegmenter) and routes each span to the matching SegmentationStrategy.

Strategy registry (priority):

1. Custom strategy registered in _TOKENIZER_REGISTRY under the script value string (e.g. "japanese", "arabic").

2. Built-in strategy for the detected ScriptType.

3. GraphemeClusterStrategy — unconditional fallback.

Parameters:

configWritingSystemAdapterConfig or None, optional

Adapter-level configuration. Default None → WritingSystemAdapterConfig() (word-level, no max size).

extra_strategiesdict[str, SegmentationStrategy] or None, optional

Additional {script_value: strategy} mappings that override the built-in registry for this adapter instance only. Useful for testing or domain-specific customisation without touching the global registry.

Parameters:

• config (WritingSystemAdapterConfig | None)

• extra_strategies (dict[str, SegmentationStrategy] | None)

Notes

User note: Use WritingSystemAdapter as the adapter parameter of SemanticChunker (Layer 3). You can also call it standalone:

adapter = WritingSystemAdapter(WritingSystemAdapterConfig(unit="word"))
spans = ScriptSegmenter().segment(text)
chunks = adapter.adapt(spans)

Developer note (Q3 compliance): Every strategy that cannot load its optional dependency logs a warning and falls back to GraphemeClusterStrategy. This adapter additionally wraps every strategy.segment() call in a try/except so a bug in a custom strategy cannot crash the pipeline.

Examples

Try it in your browser!

>>> adapter = WritingSystemAdapter()
>>> spans = ScriptSegmenter().segment("Hello 世界")
>>> chunks = adapter.adapt(spans)
>>> [c.text for c in chunks]
['Hello', '世', '界']

Go BackOpen In Tab

adapt(spans)[source]#

Segment spans using the per-script strategy registry.

Parameters:

spanssequence of ScriptSpan

Script spans from ScriptSegmenter.

Returns:

list[Chunk]

Flat, ordered list of all chunks across all spans. Always non-empty for non-empty input.

Parameters:

spans (Sequence[ScriptSpan])

Return type:

list[Chunk]

adapt_text(text)[source]#

Use convenience method: segment plain text end-to-end.

Runs ScriptSegmenter internally, then calls adapt.

Parameters:

textstr

NFC-normalised input text.

Returns:

list[Chunk]

Flat ordered chunk list.

Parameters:

text (str)

Return type:

list[Chunk]