Statistics › Section J · Hypothesis testing with two samples › Topic J.4
Comparing two proportions
Does the vaccine cut infections? Does version B convert better? Comparing two proportions is the trial, the A/B test and the survey comparison in one recipe - with one new twist: under the null, the two samples share a single success rate, and the test's standard error is built from it.
Rates in competition
Clinical trials compare infection rates between vaccine and placebo groups; websites compare purchase rates between two checkout designs; surveys compare approval between two regions. The parameter is a difference of proportions, \(p_1 - p_2\); the null of no difference reads \(H_0: p_1 = p_2\); and the machinery is the familiar difference-over-standard-error shape with one genuinely new ingredient: the pooled proportion. Two of this course's biggest application areas - experimental medicine and online experimentation - run on exactly this test, so the details deserve their own topic.
The pooled proportion
Here is the subtle step. If \(H_0: p_1 = p_2\) holds, then both populations share one success rate - call it \(p\) - and the best estimate of that single rate combines both samples:
the total successes over the total trials. This pooled \(\hat{p}\) plays the role that \(p_0\) played in the one-proportion test: the null's world has one rate, and the standard error must describe that world. It is the p0-in-the-SE principle, one level up - one-proportion tests assumed a claimed value; two-proportion tests estimate the shared value from all the data.
The two-proportion z statistic
The test statistic compares the difference in sample proportions against the pooled standard error:
with \(\hat{p}\) the pooled rate from both samples. Variances still add - the SE combines the two samples' shares of blur through the \(1/n\) weights. The reference curve is the standard normal, licensed by large-enough counts.
Conditions: counts in both samples
The conditions extend the one-proportion rules to both groups. Two independent random samples - and genuinely independent of each other: different people, randomised or disjoint. The 10% rule for each sample where populations are finite. And large enough counts: at least 10 successes and 10 failures in each sample, judged with the pooled rate for a test. The count condition is what licenses the normal curve; with rare events or small groups, exact methods or a different design are the honest answer, and a statistician's early judgement call is often simply whether the counts exist to support the comparison at all.
Worked example: the vaccine trial
In a randomised trial, 15 of 200 vaccinated participants are infected over a season, against 30 of 200 unvaccinated. Do the infection rates differ? 1. \(H_0: p_1 = p_2\); \(H_a: p_1 \ne p_2\) - two-sided, as the question names no direction. 2. Pooled rate: \(\hat{p} = 45/400 = 0.1125\). 3. The pooled standard error:
4. \(z = \frac{0.075 - 0.15}{0.0316} \approx -2.37\); two-sided \(p \approx 0.018\). 5. \(p \le 0.05\): reject - convincing evidence of a difference, with the vaccinated group's infection rate lower.
Worked example: the companion interval
The 95% interval for the difference does NOT use the pooled rate - a confidence interval assumes no null, so each sample supplies its own spread:
1. Estimate: \(0.075 - 0.15 = -0.075\). 2. Margin: \(1.96 \times 0.0314 \approx 0.061\). 3. Interval: \((-0.136, -0.014)\). 4. Interpretation: we are 95% confident the vaccine reduces the infection rate by between about 1.4 and 13.6 percentage points. Check: the interval excludes 0, agreeing with the test - pooled for the test, unpooled for the interval, and both consistent.
Pooled for the test, unpooled for the interval
The two-proportion case doubles the one-proportion lesson. A test lives in the null's world, so its SE uses the null's rate - here pooled from both samples, since the null claims one shared rate. An interval assumes nothing, so its SE uses each sample's own \(\hat{p}\). Mixing them up is the most common two-proportion error: a pooled interval borrows an assumption it has no right to, and an unpooled test describes a world other than the null's. The rule to keep: if the calculation assumes \(H_0\), pool; if it assumes nothing, don't. The two SEs usually differ only slightly (0.0316 against 0.0314 here), so interval and test almost always agree - but because they are not identical, a borderline case can occasionally split them.
Common mistakes to avoid
First, building the test's SE from the two separate sample rates - pool under the null. Second, building the interval's SE from the pooled rate - the interval owes the null nothing. Third, skipping the count check: 15 infections clears 10, but a rarer outcome in a smaller trial might not. Fourth, comparing groups that are not independent - the same people asked before and after is a paired design, not two samples. Fifth, quoting only the p-value: "the vaccine works (p = 0.018)" hides that the plausible reduction runs from 1.4 to 13.6 points. Sixth, letting unequal sample sizes cause panic - the SE's \(1/n\) weights handle them.
Key takeaways
Two-proportion comparisons test \(H_0: p_1 = p_2\) with \(z = (\hat{p}_1 - \hat{p}_2)/\sqrt{\hat{p}(1-\hat{p})(1/n_1 + 1/n_2)}\), where \(\hat{p}\) pools both samples' successes - the null's single shared rate, estimated from everything. Conditions: independent random samples, the 10% rule, and 10 successes-and-failures per sample. The vaccine trial: pooled \(\hat{p} = 0.1125\), SE 0.0316, \(z \approx -2.37\), \(p \approx 0.018\) - reject; the unpooled interval runs from \(-0.136\) to \(-0.014\), a 1.4-to-13.6-point reduction. Pooled for tests, unpooled for intervals - and always report the size.
Flashcards
Flashcard
The no-difference null for two rates
Test your understanding
Question 1 of 5
A trial has 15 infections among 200 vaccinated and 30 among 200 unvaccinated. What is the pooled proportion?
Question 2 of 5
With pooled p̂ = 0.1125 and n1 = n2 = 200, the test's standard error is about:
Question 3 of 5
With p̂1 = 0.075, p̂2 = 0.15 and SE ≈ 0.0316, the test statistic is about:
Question 4 of 5
In the TEST's standard error, which success rate appears?
Question 5 of 5
The 95% interval for the difference is (−0.136, −0.014). At α = 0.05, the test's decision will be:
