🧮 Sample size

Sample size#

The number of observations collected; larger samples shrink estimation error.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Sample size \(n\) is the number of observations used to estimate a parameter or test a hypothesis. It is the master dial of inference, setting the precision of estimates, the power of tests, and the stability of conclusions — in a word, the resolution of what the data can tell you.

The square-root law#

The central relationship is

\[\text{SE} \propto \frac{1}{\sqrt{n}}.\]

The square root has a sharp consequence: doubling \(n\) does not halve the standard error — to halve it you must quadruple \(n\). Precision is bought at an accelerating price.

In estimation and testing#

For estimation, larger \(n\) means narrower confidence intervals. For testing, the statistic is roughly \(\text{effect}/\text{SE}\), so as the SE shrinks the statistic grows — the same effect becomes easier to push past the critical value, and power rises. Small effects only become detectable once \(n\) is large enough.

The trade-off with effect size#

Sample size and effect size substitute for one another: a large effect shows up in a small sample, while a small effect needs a large one. In short, sample size compensates for a weak signal — which is exactly why it is chosen up front via power analysis from \(\alpha\), target power, the expected effect and the variance.

What n cannot do#

More data cannot fix bias, poor measurement or a wrong model, and it can make trivial effects statistically significant — precision is not correctness or importance. So interpretation, not just design, depends on \(n\): a non-significant result with small \(n\) is inconclusive, and a significant one with enormous \(n\) should be checked for practical relevance. The emphasis shifts by context — precision in estimation, power in testing, generalisation in ML, confounding in observational work — but the concept is one.

Theme: Statistical Inference & Power · All terminology

Hint

Mind map — connected ideas

Standard Error (SE) · Statistical Power · A Priori Power Analysis · Effect Size (δ) · Power (1 – β) · Trivial Effects

Hint

More in Statistical Inference & Power

A Priori Power Analysis · Chi-square (χ²) Test · Clopper–Pearson Interval · Compromise Power Analysis · Confidence Intervals (CIs) · Effect Size (δ) · Hypothesis Testing · Kolmogorov–Smirnov (KS) Test · Minimum Detectable Lift (MDL) · P-Value (probability value) · Post Hoc Power Analysis · Power (1 – β) · Power Analysis · Significance Level (α)