Statistically Significant#
Describing a result unlikely under the null hypothesis at the chosen level.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
A result is statistically significant when the observed effect is unlikely to have arisen by chance under the null hypothesis, judged against a chosen significance level \(\alpha\). Operationally it means there is enough evidence to reject \(H_0\) — and crucially, “significant” here means statistical evidence, not real-world importance.
The decision rule#
It comes down to comparing the p-value to the threshold: if \(p \le \alpha\), the result is statistically significant and you reject \(H_0\); if \(p > \alpha\), it is not, and you fail to reject. The usual \(\alpha\) is 0.05.
Examples#
A drug trial with \(p = 0.01\) against \(\alpha = 0.05\) is significant — evidence the drug beats placebo. An A/B test where a new button lifts clicks 3% but returns \(p = 0.2\) is not significant — the lift could be noise.
The cautions#
Three matter. Significance is not importance: with a large enough dataset a trivial 0.5% effect can clear the bar yet mean nothing. It is sample-size dependent: bigger samples make significance easier to reach. And it is vulnerable to p-hacking — running many tests or slicing data until something crosses \(\alpha\). A significant result is a starting point for judgement, read alongside effect size and context, not a verdict on its own.
Theme: Probability & Statistics Foundations · All terminology
Hint
Mind map — connected ideas
Statistical Significance · P-Value (probability value) · Significance Level (α) · Hypothesis Testing · Type I Error · Effect Size (δ)
Hint
More in Probability & Statistics Foundations
Beta Distribution · Confidence Level · Correlation · Critical Value · Cumulative Distribution Function (CDF) · Frequentist · IID (Independent and Identically Distributed) · Likelihood · Margin of Error (MoE) · Mean · Median · Normal Distribution · Outlier · Population Proportion
See also
Source article Adapted (context, re-expressed) in our own words from: Statistically Significant (insightful-data-lab.com).