EmbeddingManifest#

class scikitplot.corpus.EmbeddingManifest(provider, model, revision=None, dimension=None, dtype='float32', normalization='none', preprocessing=None, extra=<factory>)[source]#

Identity of one embedding generation.

Parameters:

providerstr: Backend that produced the vectors, e.g. "sentence_transformers", "openai", "custom".
modelstr: Model identifier as the provider names it.
revisionstr or None, optional: Model revision/commit, when the provider exposes one. None means unpinned, which is recorded rather than hidden: two unpinned generations of the same model are not assumed identical.
dimensionint or None, optional: Vector length. None until the first vector is produced.
dtypestr, optional: Element type, e.g. "float32".
normalizationstr, optional: "l2", "none", or a provider-specific scheme.
preprocessingstr or None, optional: Identifier for the text transformation applied before embedding.
extradict, optional: Additional JSON-compatible provenance.

Parameters:

provider (str)
model (str)
revision (str | None)
dimension (int | None)
dtype (str)
normalization (str)
preprocessing (str | None)
extra (dict[str, Any])

Notes

Developer. frozen=True because a manifest describes vectors that already exist. A mutable manifest could drift away from the vectors it claims to describe, which is the failure it exists to prevent.

Examples

>>> a = EmbeddingManifest(provider="p", model="m", dimension=8)
>>> b = EmbeddingManifest(provider="p", model="m", dimension=8)
>>> a.fingerprint == b.fingerprint
True
>>> c = EmbeddingManifest(provider="p", model="other", dimension=8)
>>> a.is_compatible(c)
False

describe()[source]#

Return a compact human-readable identifier.

Return type:: str

dimension: int | None = None#

dtype: str = 'float32'#

extra: dict[str, Any][source]#

property fingerprint: str#

Stable content-derived identifier for this generation.

Returns:

str: 16 hex characters of a SHA-256 over every semantically relevant field, length-prefixed.

Notes

Developer. This value is the embedding_model_id component of the content-derived generation identifier: a change here is exactly a change that should invalidate a built index. Deriving both from one function means they cannot disagree.

extra participates, sorted by key, so provider-specific provenance that changes the vectors also changes the fingerprint.

classmethod from_dict(data)[source]#

Rebuild from to_dict output.

Raises:

ValueError: If the payload carries a fingerprint that does not match the fields it accompanies – which means the payload was edited or corrupted.

Parameters:

data (dict[str, Any])

Return type:

EmbeddingManifest

is_compatible(other, *, assume_unpinned_match=False)[source]#

Whether vectors from self and other may share an index.

Parameters:

otherEmbeddingManifest: Manifest to compare against.
assume_unpinned_matchbool, optional: Accept two manifests that agree on every field but have no resolved revision. Default False: an unresolved revision is an unverified space, and this is the caller opting into it explicitly.

Returns:

bool: True when the two manifests describe the same embedding space.

Parameters:

other (EmbeddingManifest)
assume_unpinned_match (bool)

Return type:

bool

Notes

User. If this returns False for two manifests you believe are the same model, check whether revision is set on both. Without it there is nothing recording that the same weights produced both sets of vectors; pin the revision, or pass assume_unpinned_match=True to say you accept that.

Developer. Compatibility is fingerprint equality, deliberately. A looser rule – “same dimension and model” – would readmit the exact defect: two generations of one model with different normalization produce same-shaped vectors whose distances mean different things.

Fingerprint equality alone was not enough, though: two manifests with revision=None have equal fingerprints and so compared compatible, which reported as verified something nothing had verified. An unresolved revision now fails the check unless the caller opts in, so the unverified case is a decision someone made rather than an answer the code gave by default.

model: str[source]#

normalization: str = 'none'#

preprocessing: str | None = None#

provider: str[source]#

require_compatible(other, *, assume_unpinned_match=False)[source]#

Raise unless other is compatible with self.

Parameters:

otherEmbeddingManifest: Manifest to compare against.
assume_unpinned_matchbool, optional: Forwarded to is_compatible.

Raises:

IncompatibleEmbeddingsError: Naming both generations and either the fields that differ or, when the fields agree, the unresolved revision that prevents the match from being verified. The two cases are distinguished because they need different fixes: one is a genuine mismatch, the other is missing provenance.

Parameters:

other (EmbeddingManifest)
assume_unpinned_match (bool)

Return type:

None

revision: str | None = None#

to_dict()[source]#

Return a JSON-compatible mapping, including the fingerprint.

Return type:: dict[str, Any]

with_dimension(dimension)[source]#

Return a copy with dimension set.

Notes

Developer. Dimension is often unknown until the first vector is produced. Because it participates in the fingerprint, filling it in yields a different generation identity – which is correct: a manifest that does not yet know its dimension has not described any vectors.

Parameters:: dimension (int)
Return type:: EmbeddingManifest