Statistical Tests#
Procedures for deciding whether data support a hypothesis.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
A statistical test is a formal procedure for deciding whether data provide enough evidence to reject a default assumption. Every test follows the same five steps: state a null (H₀) and alternative (Hₐ) hypothesis, pick a significance level α, compute a test statistic, find its p-value, and interpret.
The decision rule#
Reject H₀ when p < α (the data would be surprising if H₀ were true), otherwise fail to reject it. Crucially, failing to reject is not proof that H₀ is true — absence of evidence is not evidence of absence. Two errors are possible: Type I (rejecting a true H₀, rate α) and Type II (missing a real effect, rate β).
The families#
Tests split into parametric (assuming a distribution — t-test, ANOVA) and non-parametric (assumption-free — KS, chi-square), and into one- vs two-sided. The right test depends on the data type, the question, and the assumptions you can defend.
Theme: Statistical Inference & Power · All terminology
Hint
Mind map — connected ideas
Chi-square (χ²) Test · Kolmogorov–Smirnov (KS) Test · Power Analysis · Statistical Power · Confidence Intervals (CIs) · A/B Testing
Hint
More in Statistical Inference & Power
A Priori Power Analysis · Chi-square (χ²) Test · Clopper–Pearson Interval · Compromise Power Analysis · Confidence Intervals (CIs) · Effect Size (δ) · Hypothesis Testing · Kolmogorov–Smirnov (KS) Test · Minimum Detectable Lift (MDL) · P-Value (probability value) · Post Hoc Power Analysis · Power (1 – β) · Power Analysis · Sample size
See also
Source article Adapted (context, re-expressed) in our own words from: Statistical Tests (insightful-data-lab.com).