BM25 lexical ranking#
scikitplot.rank_bm25 provides in-process lexical ranking with Okapi BM25, BM25L and BM25Plus. It makes tokenization policy, stable document identity, sparse retrieval, persistence and recipe identity explicit rather than hiding them behind a generic search interface.
Important
Text analysis is part of index correctness. Tokenize documents and queries with the same policy, bind rows to stable doc_ids when results leave the index process, and protect saved lexical state according to the sensitivity of the indexed vocabulary.
Build an index with stable doc_ids and retrieve lexical matches.
Choose Okapi/BM25L/BM25Plus and keep document/query analysis consistent.
Use sparse top-id retrieval, dense scores, stable ties, and candidate diagnostics correctly.
Save generation-based state, understand build identity, and protect derived lexical artifacts.
Sparse postings, dense vectors, tokenizer workers, and when to move to a larger search system.
BM25 classes, recipe/identity properties, validation behavior, and internal analyzer boundary.
Tokenization mismatch, OOV queries, doc-id binding, persistence, and parameter validation.
Which page should I use?#
Start with Getting started. Then choose the page that owns the decision you are making: architecture and lifecycle, configuration, persistence, performance, public API, or limits. The guide intentionally separates those concerns so examples do not accidentally become policy.
Examples#
See Rank-BM25 for executable examples that accompany this guide.