🧷 Cross-Validation (CV)

Cross-Validation (CV)#

Estimating generalisation by repeatedly training and testing on different splits.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Cross-validation (CV) is the umbrella model-evaluation technique: split the data into multiple subsets, rotate which subset is held out for testing, and average the results so the performance estimate does not hinge on a single lucky or unlucky split. Its goal is to gauge how well a model generalises to unseen data, and it underpins model selection, hyperparameter tuning and overfitting prevention.

How it works#

Partition into k folds; for each, train on the rest and test on the held-out fold; repeat so every fold serves as test once; average across folds for the final estimate.

The main flavours#

k-fold is the balanced default; stratified k-fold holds class proportions steady for imbalanced classification; leave-one-out (LOOCV) uses one sample per fold — accurate but costly; time-series CV (rolling or expanding windows) respects time order for sequential data; and nested CV wraps an inner tuning loop inside an outer evaluation loop so that hyperparameter selection does not leak into the performance estimate.

Why it matters, and the costs#

CV gives a robust, lower-variance read on generalisation and is the standard harness for grid, random and Bayesian hyperparameter search. The price is compute (the model is trained many times) and the need to match the fold type to the data — most importantly, never shuffling a time series.

Theme: Validation & Cross-Validation · All terminology

Hint

Mind map — connected ideas

k-fold cross-validation · Stratified Group K-Fold · Blocked Splits (Single Holdout) · Sliding Window (Rolling Window) Cross-Validation · Expanding Window Cross-Validation · Data Leakage

Hint

More in Validation & Cross-Validation

Blocked Splits (Single Holdout) · Data Leakage · Evaluation Set · Expanding Window Cross-Validation · k-fold cross-validation · k-fold Stratified Cross-Validation (Stratified CV) · Multiclass stratified CV · Sliding Window (Rolling Window) Cross-Validation · Stratified Group K-Fold · Stratified Shuffle Split · Time-based splits (a.k.a. Temporal Cross-Validation, Rolling Window Validation)