🧮 Significance Level (α)

Significance Level (α)#

The tolerated false-positive probability, fixed before testing.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

The significance level \(\alpha\) is the threshold probability for rejecting the null hypothesis \(H_0\) — the maximum risk of a Type I error (rejecting a true \(H_0\)) you are willing to accept. Common choices are \(\alpha = 0.05\) (the default), \(0.01\) (stricter, stronger evidence demanded) and \(0.10\) (more lenient).

The decision rule#

Compute a test statistic and its p-value, then compare: if \(p \le \alpha\), reject \(H_0\); if \(p > \alpha\), fail to reject. So \(\alpha\) is simply the decision cutoff fixed in advance.

What α is#

It is the long-run false-positive rate: at \(\alpha = 0.05\), about 5 in 100 tests of a true null will wrongly reject it. Choose it by stakes — 0.01 in medicine, genetics and other high-stakes settings; 0.05 as a general balance; 0.10 in exploratory work where missing a real effect costs more than a false alarm.

Tied to the confidence level#

Significance and confidence are complements: the confidence level is \(1 - \alpha\). An \(\alpha = 0.05\) test corresponds to 95% confidence — across repeated experiments, 95% of the resulting intervals would contain the true parameter.

What it is not#

Three cautions: \(\alpha\) is not the probability that \(H_0\) is true; it is chosen before the data, not tuned after; and clearing it means statistically significant, not practically important — for that you still need an effect size.

Theme: Statistical Inference & Power · All terminology

Hint

Mind map — connected ideas

Type I Error · P-Value (probability value) · Statistical Significance · Power (1 – β) · Critical Value · Effect Size (δ)

Hint

More in Statistical Inference & Power

A Priori Power Analysis · Chi-square (χ²) Test · Clopper–Pearson Interval · Compromise Power Analysis · Confidence Intervals (CIs) · Effect Size (δ) · Hypothesis Testing · Kolmogorov–Smirnov (KS) Test · Minimum Detectable Lift (MDL) · P-Value (probability value) · Post Hoc Power Analysis · Power (1 – β) · Power Analysis · Sample size