SQLiteStorage#

class scikitplot.corpus.SQLiteStorage(db_path=':memory:')[source]#

SQLite-backed corpus store with FTS5 full-text search.

Uses stdlib sqlite3 — no external dependencies. Full-text search is available via FTS5 (StorageQuery.full_text).

The database uses WAL (Write-Ahead Logging) mode for better concurrent read throughput. A single connection is held per SQLiteStorage instance; use separate instances for multiple threads if needed.

Parameters:

db_pathpathlib.Path or str

Path to the SQLite database file. Created if absent. Pass \":memory:\" for a purely in-memory database.

Parameters:

db_path (pathlib.Path | str)

Examples

Try it in your browser!

>>> with SQLiteStorage(":memory:") as store:
...     store.count()
0

Go BackOpen In Tab

close()[source]#

Close the database connection; safe to call more than once.

Notes

Developer. sqlite3.Connection.close is itself idempotent; any call that needs the connection afterwards raises sqlite3.ProgrammingError (“Cannot operate on a closed database”).

Return type:

None

count()[source]#

Return total stored document count via fast SQL COUNT.

Return type:

int

get(doc_id)[source]#

Retrieve a document by doc_id.

Parameters:

doc_idstr

Parameters:

doc_id (str)

Return type:

CorpusDocument | None

query(q)[source]#

Query documents with optional full-text search (FTS5).

Parameters:

qStorageQuery

Parameters:

q (StorageQuery)

Return type:

QueryResult

save(doc)[source]#

Persist a single document (upsert by doc_id).

Parameters:

docCorpusDocument

Parameters:

doc (CorpusDocument)

Return type:

None

save_batch(docs)[source]#

Persist a batch of documents atomically (all-or-nothing).

All rows are serialized and validated before any write; the documents and FTS rows are then committed inside one explicit transaction, so a failure at any point leaves the database exactly as before the batch (CORPUS-STO-001). If a doc_id occurs more than once, the final occurrence wins consistently in both tables.

Parameters:

docssequence of CorpusDocument

Parameters:

docs (Sequence[CorpusDocument])

Return type:

None

search_text(text, limit=10, *, collection_id=None)[source]#

Rank documents against free text with BM25, best first.

Parameters:

textstr

Free text: a question, an identifier, an error message. Never interpreted as FTS5 syntax.

limitint, optional

Most results to return; must be a positive integer. Default 10.

collection_idstr or None, optional

Restrict to one collection. Default: None.

Returns:

list of (CorpusDocument, float)

Each document with its BM25 score, higher meaning more relevant. Empty when text has no word or nothing matches.

Raises:

TypeError

If text is not a string or limit is not an integer.

ValueError

If limit is below one.

RuntimeError

If this SQLite build has no FTS5, so no BM25 ranking exists.

Parameters:

• text (str)

• limit (int)

• collection_id (str | None)

Return type:

list[tuple[CorpusDocument, float]]

Notes

User. Unlike StorageQuery.full_text — a filter that asks for the whole string — this is ranked retrieval: a document containing some of the words is returned, ordered by how strongly it matches. That is what a lexical leg of hybrid search needs; what is roc_auc_score() finds the document about roc_auc_score.

Developer. Built from _fts5_any, so it is never an FTS5 syntax error (finding CX-01). The score is -bm25(corpus_fts): SQLite reports stronger matches as more negative, and a leg that feeds fusion or a reader should see “higher is better”.