PreferenceEvaluation#

class scikitplot.corpus.PreferenceEvaluation(names, summary, items, judges, agreement, settings, metadata=<factory>)[source]#

Result of evaluate_preference.

Parameters:
namestuple of str

The two systems, (a, b).

summarymapping

Per judge and for the panel: wins, ties, losses (for a), win_rate ((wins + ties / 2) / judged), position_consistency and judged.

itemstuple of mapping

Per item: item_id, question, the panel score for a and, per judge, the JudgeResult as a mapping.

judgesmapping

Judge identities.

agreementfloat

Krippendorff’s alpha (nominal) over the judges’ winners; NaN with fewer than two judges or when undefined.

settings, metadatamapping
Parameters:
agreement: float[source]#
dumps(format='json')[source]#

Serialise to "json", "jsonl" (one row per item), "yaml" or "toml".

Parameters:

format (str)

Return type:

str

classmethod from_dict(data)[source]#

Rebuild a report from to_dict output.

Parameters:

data (Mapping[str, Any])

Return type:

PreferenceEvaluation

items: tuple[source]#
judges: Mapping[str, Any][source]#
metadata: Mapping[str, Any][source]#
names: tuple[source]#
classmethod read(path, format=None)[source]#

Read a report written by write in JSON, YAML or TOML.

Parameters:

  • path (Any)

  • format (str | None)

Return type:

PreferenceEvaluation

settings: Mapping[str, Any][source]#
summary: Mapping[str, Mapping[str, Any]][source]#
to_dict()[source]#

Return the full report as a mapping (schema versioned).

Return type:

dict[str, Any]

to_rows()[source]#

One row per item: identifier, panel score for a, each judge’s winner.

Return type:

list[dict[str, Any]]

write(path, format=None)[source]#

Write atomically; the format defaults to the path suffix.

Parameters:

  • path (Any)

  • format (str | None)

Return type:

Any