Two-Proportion Z-Test#
A hypothesis test for whether two groups’ success proportions differ, using a normal approximation.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
The two-proportion z-test decides whether an outcome’s rate differs significantly between two independent groups — the standard test behind A/B experiments (is control’s 5% really below treatment’s 6.2%, or just noise?).
Hypotheses#
The null is equality, \(H_0 : p_1 = p_2\); the alternative is \(H_1 : p_1 \neq p_2\) (two-tailed) or a one-sided version.
The statistic#
where \(\hat{p}_i = x_i / n_i\) are the group proportions and \(\hat{p} = (x_1 + x_2)/(n_1 + n_2)\) is the pooled proportion — the shared rate assumed under \(H_0\), used to build the standard error. Compare \(z\) to a critical value (\(\pm 1.96\) at \(\alpha = 0.05\)) or convert it to a p-value.
Worked example#
Control: 100 of 1,000 → \(\hat{p}_1 = 0.10\). Treatment: 130 of 1,000 → \(\hat{p}_2 = 0.13\). Pooled \(\hat{p} = 230/2000 = 0.115\); standard error \(\sqrt{0.115 \times 0.885 \times 0.002} \approx 0.01425\); so \(z = (0.10 - 0.13)/0.01425 \approx -2.11\), a two-tailed \(p \approx 0.035\). Since \(p < 0.05\), reject \(H_0\) — treatment converts significantly higher.
Assumptions#
Independent samples, binary (success/failure) observations, and samples large enough for the normal approximation (\(np \ge 5\) and \(n(1 - p) \ge 5\)). It powers A/B tests, clinical recovery-rate comparisons and survey yes/no contrasts alike.
Theme: Statistical Inference & Power · All terminology
Hint
Mind map — connected ideas
Proportion · Z-Score · Conversion Rate Uplift · True Conversion Rate · Traditional A/B Test (Fixed-Horizon A/B Test) · Statistical Significance
Hint
More in Statistical Inference & Power
A Priori Power Analysis · Chi-square (χ²) Test · Clopper–Pearson Interval · Compromise Power Analysis · Confidence Intervals (CIs) · Effect Size (δ) · Hypothesis Testing · Kolmogorov–Smirnov (KS) Test · Minimum Detectable Lift (MDL) · P-Value (probability value) · Post Hoc Power Analysis · Power (1 – β) · Power Analysis · Sample size
See also
Source article Adapted (context, re-expressed) in our own words from: Two-Proportion Z-Test (insightful-data-lab.com).