🧮  Trivial Effects

Trivial Effects#

Effects too small to matter in practice even if statistically detectable.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

A trivial effect is a result that is statistically significant (\(p < \alpha\)) yet so small in magnitude that it has little or no practical importance. The difference is real — not chance — but too tiny to matter.

Why they appear#

Two causes. First, large samples: with enough data, even a minuscule difference clears the significance bar — in an A/B test on millions of users, a 0.05% conversion bump can be “significant” yet meaningless. Second, fixating on p-values: a p-value says whether an effect exists, not how big it is, so reading it without an effect size invites over-interpretation.

Examples#

A drug that lowers blood pressure by 0.5 mmHg versus standard care, tested on 10,000 patients, can show \(p < 0.001\) — significant, but clinically trivial. A landing page that moves conversion from 10.00% to 10.05% across 1,000,000 users gives \(p < 0.01\), yet a 0.05% lift is not worth deploying.

How to avoid the trap#

Always report an effect size (Cohen’s d, a difference in proportions) next to the p-value; read the confidence interval (a tight band hugging zero signals triviality); judge practical relevance; and fix a minimum detectable effect in advance — “we act only if conversion improves by at least +1%.”