SemantemeInfo#

class scikitplot.corpus.SemantemeInfo(surface, script=None, direction=None, morphemes=None, lemma=None, pos_tag=None, stem=None, grapheme_count=None, codepoint_count=None, is_stopword=None, language_hint=None, determinative=None, raw_surface=None, embedding=None)[source]#

Rich semantic unit descriptor attached to each multilang chunk.

A semanteme is the smallest unit of meaning in a language. Depending on the script and chunking backend this may be a morpheme, a syllabic aksara, a Han ideograph, or a grapheme cluster.

All fields are optional so that callers that do not have the full analysis available can still populate a partial record.

Parameters:
surfacestr

The surface form as it appears in the normalised text.

scriptstr or None

ScriptType value string of this semanteme (e.g. "latin", "han").

directionstr or None

Writing direction: "ltr", "rtl", or "ttb".

morphemeslist[str] or None

Morpheme decomposition (e.g. MeCab or camel-tools output). None when morphological analysis was not run.

lemmastr or None

Dictionary lemma of this semanteme. None if unavailable.

pos_tagstr or None

Part-of-speech tag (language-specific tagset string).

stemstr or None

Stemmed form (Porter, Snowball, etc.).

grapheme_countint or None

Number of UAX #29 grapheme clusters in surface.

codepoint_countint or None

Number of Unicode codepoints in surface.

is_stopwordbool or None

True if this semanteme is a stopword in its detected language.

language_hintstr or None

ISO 639-1 / BCP-47 language code hint (e.g. "ja", "ar").

determinativestr or None

For Egyptian hieroglyphs: the determinative sign glyph.

raw_surfacestr or None

Pre-normalisation surface form. None when no normalisation changed the text (idempotent normalisation).

embeddinglist[float] or None

Per-semanteme dense embedding vector (if computed). Allows fine-grained semantic search at morpheme/aksara level.

Parameters:

  • surface (str)

  • script (str | None)

  • direction (str | None)

  • morphemes (list[str] | None)

  • lemma (str | None)

  • pos_tag (str | None)

  • stem (str | None)

  • grapheme_count (int | None)

  • codepoint_count (int | None)

  • is_stopword (bool | None)

  • language_hint (str | None)

  • determinative (str | None)

  • raw_surface (str | None)

  • embedding (list[float] | None)

Notes

Developer note: SemantemeInfo is stored as a list in MultilangChunkMeta.semantemes and serialised to CorpusDocument.morphemes (list of surface strings) and chunk.metadata["semantemes"] (list of dicts).

Examples

>>> info = SemantemeInfo(
...     surface="run", script="latin", lemma="run", stem="run", morphemes=["run"]
... )
>>> info.surface
'run'
codepoint_count: int | None = None#
determinative: str | None = None#
direction: str | None = None#
embedding: list[float] | None = None#
grapheme_count: int | None = None#
is_stopword: bool | None = None#
language_hint: str | None = None#
lemma: str | None = None#
morphemes: list[str] | None = None#
pos_tag: str | None = None#
raw_surface: str | None = None#
script: str | None = None#
stem: str | None = None#
surface: str[source]#
to_dict()[source]#

Serialise to a JSON-safe dict for chunk.metadata storage.

Returns:
MetadataDict

All fields; fields that are None are omitted to minimise payload size.

Return type:

dict[str, Any]

with_embedding(vector)[source]#

Return a copy of this record with an embedding vector attached.

Parameters:
vectorlist[float]

Dense embedding from any encoder (sentence-transformers, etc.).

Returns:
SemantemeInfo

New frozen instance.

Parameters:

vector (list[float])

Return type:

SemantemeInfo

Notes

CRITICAL-04 fix: uses dataclasses.replace so that any field added to SemantemeInfo in the future is automatically preserved without updating this method.