What Are Statistical Tests?#

Stage 2 · 🔗 Associations & Correlation · Lesson 15 of 56 · beginner

◀ Previous · Harald Cramér · Next · Eta Squared (η²): Effect Size in ANOVA ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

Is it real or chance?#

When you measure an association in a sample, one question always lurks: is it real, or could it be chance? A correlation of 0.2 in a handful of taxi trips might vanish in the next batch. Statistical tests answer this — they quantify how likely your finding is to have arisen by luck alone.

Null and alternative#

A test pits two hypotheses against each other. The null hypothesis (\(H_0\)) is the sceptic’s position: no effect, no association — any pattern seen is chance. The alternative (\(H_1\)) is the claim you are investigating: a real effect exists. The test computes a test statistic from the data, measuring how far the sample departs from what \(H_0\) would predict.

The p-value#

That departure is summarised in a p-value: the probability of seeing data at least as extreme as yours if the null hypothesis were true. A small p-value means your result would be surprising under “pure chance”, so chance is an unconvincing explanation; a large one means the data is unremarkable and the null stands. You compare it to a chosen threshold, the significance level \(\alpha\) (commonly 0.05), and reject \(H_0\) when \(p < \alpha\).

Reading the result#

Two cautions make tests trustworthy. Rejecting a true null is a false positive (a Type I error, at rate \(\alpha\)); failing to detect a real effect is a false negative (Type II). And significance is not importance — with enough data a trivially small effect becomes “significant”, which is why the next lesson pairs tests with a measure of effect size. The chi-square, t-test and ANOVA F-test are all instances of this one logic.

Hint