FeatureSchema#

class scikitplot.feature_selection.FeatureSchema(features, policy, n_rows)[source]#

Immutable, fitted description of a table’s columns.

Parameters:

featurestuple of FeatureSpec: One spec per column, in training column order.
policyFeaturePolicy: The policy the schema was fitted with.
n_rowsint: Training rows.

Parameters:

features (tuple[FeatureSpec, ...])
policy (FeaturePolicy)
n_rows (int)

See also

profile_features: Fit a schema.
FeatureProfiler: Fit a schema inside a scikit-learn pipeline.

Notes

User note. A schema is fitted state: build it from training data once, keep it (save), and audit later data against it.

Examples

>>> import pandas as pd
>>> from scikitplot.feature_selection import profile_features
>>> X = pd.DataFrame(
...     {
...         "age": [21, 35, 47, 52, 33, 41] * 5,
...         "city": ["a", "b", "a", "c", "b", "a"] * 5,
...         "vip": [True, False] * 15,
...         "churn": [0, 1] * 15,
...     }
... )
>>> schema = profile_features(X, {"roles": {"target": ["churn"]}})
>>> schema.groups(by="kind")
{'binary': ['vip'], 'categorical_nominal': ['city'], 'numeric_discrete': ['age']}
>>> schema["age"].is_("discrete"), schema["churn"].role
(True, 'target')

assessment_table()[source]#

Return a features-by-assessments table of states.

Returns:

pandas.DataFrame: Index: feature names; columns: assessment names; values: state strings.

Return type:

DataFrame

audit(X)[source]#

Compare later data’s columns and dtypes with the fitted schema.

Parameters:

Xpandas.DataFrame: Validation, test or production data.

Returns:

pandas.DataFrame: One row per finding with columns column, issue (missing_column, extra_column, order_changed, family_changed, dtype_changed), expected, observed. Empty when the structure matches. dtype_changed with an unchanged family (int64 to Int64) is informational.

Parameters:

X (DataFrame)

Return type:

DataFrame

Notes

The schema is not changed. Distribution changes are the job of ReferenceProfile.

binning_plan()[source]#

Return the binning recommendation of every numeric feature.

Returns:

pandas.DataFrame: Columns state, rule, n_bins, rule_n_bins, strategy, edges, reasons; indexed by feature.

Return type:

DataFrame

check(X, *, allow=('dtype_changed',))[source]#

Raise when audit finds an issue that is not allowed.

Parameters:

Xpandas.DataFrame: Later data.
allowsequence of str, default=(“dtype_changed”,): Issue codes to tolerate.

Raises:

SchemaMismatchError: Listing every disallowed finding.

Parameters:

X (DataFrame)
allow (Iterable[str])

Return type:

None

dumps(format='json')[source]#

Serialise the schema (verified by reading it back).

Parameters:

format{“json”, “yaml”, “toml”}, default=”json”: Output format.

Returns:

str: The document.

Parameters:

format (str)

Return type:

str

feature(key)[source]#

Return a feature by name (or position).

Parameters:

keystr or int: Column name, or position.

Returns:

FeatureSpec: The feature.

Raises:

KeyError: If the name is unknown.

Parameters:

key (str | int)

Return type:

FeatureSpec

property feature_names: list[str]#

Names of the role="feature" columns.

Type:: List of str

features: tuple[FeatureSpec, ...][source]#

classmethod from_dict(data)[source]#

Rebuild a schema from to_dict output.

Parameters:

datamapping: Stored form.

Returns:

FeatureSchema: The schema.

Raises:

IncompatibleSchemaError: For another schema_version or artefact type.
ValueError: For an off-contract record or a policy digest that does not match the stored policy.

Parameters:

data (Mapping[str, Any])

Return type:

FeatureSchema

groups(by='kind', *, role='feature')[source]#

Group feature names.

Parameters:

by{“kind”, “family”, “role”, “tag”, “assessment”}, default=”kind”: Grouping key. "assessment" lists, per assessment, the features whose state is "true"; "tag" lists, per tag, the tagged features (a feature can be in several groups).
rolestr or sequence of str or None, default=”feature”: Only features with these roles; None for all.

Returns:

dict of str to list of str: Group name (sorted) to member names (schema order); empty groups are omitted.

Parameters:

by (str)
role (Any)

Return type:

dict[str, list[str]]

classmethod load(path, format=None)[source]#

Read a schema written by save.

Parameters:

pathstr or os.PathLike: Source file.
formatstr or None, default=None: Explicit format.

Returns:

FeatureSchema: The schema.

Parameters:

path (Any)
format (str | None)

Return type:

FeatureSchema

classmethod loads(text, format='json')[source]#

Parse a schema document.

Parameters:

textstr: The document.
format{“json”, “yaml”, “toml”}, default=”json”: Input format.

Returns:

FeatureSchema: The schema.

Parameters:

text (str)
format (str)

Return type:

FeatureSchema

n_rows: int[source]#

property names: list[str]#

Every column name, in training order.

Type:: List of str

policy: FeaturePolicy[source]#

property policy_digest: str#

Digest of the policy the schema was fitted with.

Type:: str

rename(mapping)[source]#

Return a schema whose features are renamed, keeping old names as aliases.

Parameters:

mappingmapping of str to str: Old name to new name; unlisted features keep their names.

Returns:

FeatureSchema: The renamed schema (identity, stats and assessments unchanged).

Raises:

KeyError: If a key is not a feature name.
ValueError: If the result has duplicate names.

Parameters:

mapping (Mapping[str, str])

Return type:

FeatureSchema

resolve_alias(name)[source]#

Return the current name of a feature known by name or an alias.

Parameters:

namestr: Current name or any earlier name.

Returns:

str: The current name.

Raises:

KeyError: If no feature has that name or alias.

Parameters:

name (str)

Return type:

str

save(path, format=None)[source]#

Write the schema to path atomically.

Parameters:

pathstr or os.PathLike: Target file (.json, .yaml, .toml).
formatstr or None, default=None: Explicit format.

Returns:

pathlib.Path: The written path.

Parameters:

path (Any)
format (str | None)

Return type:

Path

select(*, kind=None, family=None, role='feature', tag=None, is_=(), is_not=(), names=None)[source]#

Return the names of the features that satisfy every given condition.

Parameters:

kind, familystr or sequence of str or None, default=None: Accept features of any of these kinds / families.
rolestr or sequence of str or None, default=”feature”: Accept these roles; None accepts every role.
tagstr or sequence of str or None, default=None: Accept features carrying at least one of these tags.
is_sequence of str, default=(): Assessments that must all be "true".
is_notsequence of str, default=(): Assessments that must not be "true" (unknown passes).
namessequence of str or None, default=None: Restrict to these names.

Returns:

list of str: Matching names in schema order.

Raises:

ValueError: For an unknown kind, family, role or assessment name.

Parameters:

kind (Any)
family (Any)
role (Any)
tag (Any)
is_ (Iterable[str])
is_not (Iterable[str])
names (Iterable[str] | None)

Return type:

list[str]

Examples

>>> import pandas as pd
>>> from scikitplot.feature_selection import profile_features
>>> X = pd.DataFrame({"a": [1.5, 2.5, 3.5, 95.5] * 5, "b": list("xyzx") * 5})
>>> schema = profile_features(X)
>>> schema.select(kind="numeric_continuous", is_=("skewed",))
['a']