🔬 Bootstrap Confidence Intervals (CIs)

Bootstrap Confidence Intervals (CIs)#

Interval estimates built by resampling the data with replacement and recomputing the statistic many times.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

A bootstrap confidence interval estimates the uncertainty of a statistic — a mean, median, regression coefficient, even an AUROC — by resampling the data rather than relying on a parametric formula. The idea: when the population distribution is unknown, approximate it by drawing from the sample you already have. It shines when sample sizes are small, the data are non-normal, or no clean standard-error formula exists.

The procedure#

Start with the original sample of size \(n\).
Resample with replacement to build \(B\) bootstrap samples (often \(B = 1000\) or more), each of size \(n\).
Compute the statistic on each bootstrap sample.
The spread of those \(B\) values is the bootstrap distribution of the statistic.
Read a confidence interval off that distribution.

Three ways to build the interval#

Percentile — take the \(\alpha/2\) and \(1-\alpha/2\) quantiles directly (a 95% CI is the 2.5th–97.5th percentiles).
Basic (reverse percentile) — reflect the percentile interval around the observed statistic to correct simple bias.
BCa (bias-corrected and accelerated) — adjusts for both bias and skew in the bootstrap distribution; usually the most accurate and the default recommendation.

Worked example#

For \(X = [5, 7, 9, 10, 12, 8, 6, 7, 9, 11]\) the mean is 8.4. Draw 1000 resamples, take each mean, and read the 2.5th and 97.5th percentiles — about 7.2 and 9.6 — giving a 95% CI of [7.2, 9.6].

import numpy as np

rng = np.random.default_rng(42)
boot = [rng.choice(X, size=len(X), replace=True).mean() for _ in range(1000)]
lo, hi = np.percentile(boot, [2.5, 97.5])

Pitfalls and edge cases#

Too few resamples — small \(B\) makes the interval itself noisy; prefer thousands.
A bad sample stays bad — the bootstrap can only resample what you have; a tiny or unrepresentative sample yields a confident-looking but wrong interval.
Dependent data break it — for time series or grouped data the plain bootstrap destroys the dependence structure; use a block bootstrap instead.

Theme: Model Evaluation & Uncertainty · All terminology

Hint

Mind map — connected ideas

Subsampling · Probability · Mann–Whitney U Test (also called the Wilcoxon rank-sum test)

Hint

More in Model Evaluation & Uncertainty

Average Absolute Error (AAE) · Baseline Heuristics · Bootstrap · Coverage · Cramér’s V · DeLong’s Test · KS Statistic (Kolmogorov–Smirnov Statistic) · Likelihood Ratio (LR) · Mann–Whitney U Test (also called the Wilcoxon rank-sum test) · MASE (Mean Absolute Scaled Error) · Mean Absolute Error (MAE) · Mean Absolute Percentage Error (MAPE) · Mean Squared Error (MSE) · Relative accuracy