🧷 k-fold cross-validation

k-fold cross-validation#

Train on k-1 folds and test on the held-out fold, rotating through all k.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

k-fold cross-validation splits the data into k roughly equal folds and trains and tests the model k times, each run holding out a different fold as the test set and training on the other \(k-1\). Averaging the k scores gives a lower-variance estimate of performance than any single train/test split — which is why it is the default CV method.

How it works#

Shuffle (if order is irrelevant), split into k folds (commonly 5 or 10), and for each fold \(i\) train on the other folds and test on fold \(i\). Collect the k scores and average them for the final metric.

Example#

With 1,000 samples and k = 5, each fold is 200 samples: every run trains on 800 and validates on 200, rotating which 200 is held out, and the result is the mean across the five runs.

Variations#

Stratified k-fold preserves class balance per fold (vital for imbalanced data); repeated k-fold re-runs the whole process with new splits for a steadier estimate; and leave-one-out (LOOCV) is the extreme \(k = N\), one sample per fold — very accurate, very expensive.

In scikit-learn, and the trade-offs#

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5)
print(scores.mean(), scores.std())

The gains — a reliable estimate, less dependence on one random split, full use of the data, and a backbone for hyperparameter tuning — cost k model fits, and plain k-fold is wrong for time series, where time-aware CV is required instead.

Theme: Validation & Cross-Validation · All terminology

Hint

Mind map — connected ideas

Cross-Validation (CV) · Stratified Group K-Fold · Stratified Shuffle Split · Multiclass stratified CV · Blocked Splits (Single Holdout) · Data Leakage

Hint

More in Validation & Cross-Validation

Blocked Splits (Single Holdout) · Cross-Validation (CV) · Data Leakage · Evaluation Set · Expanding Window Cross-Validation · k-fold Stratified Cross-Validation (Stratified CV) · Multiclass stratified CV · Sliding Window (Rolling Window) Cross-Validation · Stratified Group K-Fold · Stratified Shuffle Split · Time-based splits (a.k.a. Temporal Cross-Validation, Rolling Window Validation)