SemanticChunkerBridge#
- class scikitplot.corpus.SemanticChunkerBridge(inner)[source]#
Bridge for
SemanticChunker→ChunkerBasecontract.The
SemanticChunker(Layer 3) is a new-style chunker that returnsChunkResultobjects. This bridge adapts it to theChunkerBaseinterface so it can be passed directly toCorpusPipelineandDocumentReaderwithout modification.SEMANTICis used as the strategy value. IfChunkingStrategydoes not yet define aSEMANTICmember,CUSTOMis used as a fallback so the bridge never raises anAttributeError.Notes
User note: Pass
SemanticChunker(...)directly toCorpusPipeline— the bridge is applied automatically viabridge_chunker.Developer note: Multilang metadata in
chunk.metadata["multilang"]flows throughChunkedTextListto_base.DocumentReader.get_documentswhich maps it toCorpusDocumentfields.- Parameters:
inner (Any)
- chunk(text, metadata=None)[source]#
Chunk text and return a
ChunkResult.CRITICAL-02 (Phase 2): Returns
ChunkResultdirectly.DocumentReader.get_documentsnow iterateschunk_result.chunksinstead of(char_start, chunk_text)tuples.- Parameters:
- textstr
Raw text to chunk.
- metadatadict[str, Any] or None, optional
Raw-chunk metadata dict passed by
get_documents(). Forwarded asextra_metadatato the inner chunker.
- Returns:
- ChunkResult
Ordered list of
Chunkobjects withtext,start_char,end_char, andmetadata.
- Parameters:
- Return type:
Notes
Use
_to_tuplesto convert to the legacylist[tuple[int, str]]format if needed for backward compat.
- strategy: ClassVar[ChunkingStrategy] = 'semantic'[source]#