Statistics › Section J · Hypothesis testing with two samples › Topic J.1

Comparing two means with unknown spreads

Does the new method beat the old? Do two groups differ? Comparing two means is the most common question in applied statistics, and the machinery is familiar: the statistic becomes a difference, the standard error combines both samples' spreads, and the t distribution prices the comparison.

Two groups, one question

Does the new tutoring method beat the old? Do two cities' commutes differ? The question moves from one mean to a comparison, and the parameter becomes a difference: \(\mu_1 - \mu_2\). The null of "no difference" reads \(H_0: \mu_1 = \mu_2\), or equivalently \(H_0: \mu_1 - \mu_2 = 0\); the alternative carries the question's direction: \(\mu_1 > \mu_2\), \(\mu_1 < \mu_2\), or the two-sided \(\mu_1 \ne \mu_2\). Everything from the one-sample chapter now runs on the difference.

The two-sample t statistic

Two independent random samples give \(\bar{x}_1\), \(s_1\), \(n_1\) and \(\bar{x}_2\), \(s_2\), \(n_2\). The test statistic compares the difference in sample means against the standard error of a difference:

t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}

Both spreads are estimated, so the reference curve is the \(t\) distribution; the degrees of freedom come from an approximation (software reports it; the smaller of \(n_1 - 1\) and \(n_2 - 1\) is a safe conservative round value). The formula handles unequal sample sizes naturally - each sample contributes its own \(s^2/n\), and the group measured more precisely simply pulls less blur into the comparison. Equal sample sizes are convenient, never required.

Why variances add

The denominator deserves a second look, because it is where intuition fails. Both \(\bar{x}_1\) and \(\bar{x}_2\) are estimates with their own wobble, \(s_1^2/n_1\) and \(s_2^2/n_2\) - and independent wobbles combine by adding variances, the rule from the expected-value and standard-deviation topic. Even though the statistic is a DIFFERENCE, the variances add: subtracting two independent noisy quantities is noisier than either one alone, never quieter. The standard error of the difference is therefore \(\sqrt{s_1^2/n_1 + s_2^2/n_2}\), and writing \(\sqrt{s_1^2/n_1 - s_2^2/n_2}\) is the classic error - sometimes even producing an imaginary number, which should alarm anyone.

Conditions: independent inside and out

Three conditions license the two-sample \(t\). Both samples are random. Observations are independent within each group, AND the two groups are independent of each other - no person, pair or unit appears in both samples, and no group's outcome influences the other's. Shape: each sample needs \(n \ge 30\) or roughly symmetric data, the same grading as the one-sample case. The between-group condition is the newcomer, and it is exactly what paired designs violate - deliberately, for which the matched-samples topic supplies the fix.

Worked example: two tutoring methods

Students are randomly assigned to two tutoring methods. Method A: \(n_1 = 25\), \(\bar{x}_1 = 80\), \(s_1 = 8\). Method B: \(n_2 = 30\), \(\bar{x}_2 = 74\), \(s_2 = 9\). Does A differ from B? 1. \(H_0: \mu_1 = \mu_2\); \(H_a: \mu_1 \ne \mu_2\) - two-sided. 2. Conditions: random assignment, independent groups, 30 in group B, and group A's 25 scores show no strong skew. 3.

\text{SE} = \sqrt{\frac{64}{25} + \frac{81}{30}} = \sqrt{2.56 + 2.70} = \sqrt{5.26} \approx 2.29

so \(t = \frac{80 - 74}{2.29} \approx 2.62\), with about 53 degrees of freedom. 4. Two-sided \(p \approx 0.012 \le 0.05\): reject - convincing evidence the methods differ, with A scoring higher.

Worked example: the companion interval

The same comparison as an interval: \( (\bar{x}_1 - \bar{x}_2) \pm t^* \text{SE} \), with \(t^* \approx 2.01\) for 95% at these degrees of freedom. 1. Estimate: \(80 - 74 = 6\) points. 2. Margin: \(2.01 \times 2.29 \approx 4.6\). 3. Interval: \((1.4, 10.6)\) points. 4. Interpretation: we are 95% confident method A averages between about 1.4 and 10.6 points higher than method B. Check: the interval excludes 0, agreeing with the test's rejection - and it says how large the advantage might be, from marginal to substantial, which the p-value alone cannot.

The interval-test agreement, again

The two results always agree, and the pattern is worth internalising: a 95% interval for \(\mu_1 - \mu_2\) excludes 0 exactly when the two-sided 0.05-level test rejects \(H_0: \mu_1 = \mu_2\). The interval is the richer report - it answers the test's yes-or-no AND sizes the difference in one statement. Applied statistics increasingly reports intervals first and tests second, for exactly this reason: "A scores 6 points higher, give or take about 4.6" tells a reader what "p = 0.012" cannot.

Common mistakes to avoid

First, subtracting variances in the SE: independent wobbles add, even for a difference in means. Second, treating paired data as two independent samples - the before-and-after design of the next topics needs the differences, not the two columns. Third, letting the same units appear in both groups (the same students take both methods): the groups are not independent. Fourth, quoting the smaller \(n\) as the degrees of freedom - it is the smaller \(n\) MINUS 1, conservatively, and software's value is better. Fifth, reporting the p-value without the difference and its interval. Sixth, testing "x̄1 = x̄2" - sample means are known; the hypothesis is about the parameters.

Key takeaways

Two-group comparisons test the difference \(\mu_1 - \mu_2\), with \(H_0: \mu_1 = \mu_2\) and the question's direction in \(H_a\). The two-sample \(t\) statistic is \((\bar{x}_1 - \bar{x}_2)/\sqrt{s_1^2/n_1 + s_2^2/n_2}\) - variances added, never subtracted - on the \(t\) curve with degrees of freedom from software or a conservative smaller-\(n\) rule. Conditions: random, independent within and between groups, shape. The tutoring comparison: SE 2.29, \(t \approx 2.62\), \(p \approx 0.012\) - reject; interval \((1.4, 10.6)\) points favouring A. Interval and test always agree, and the interval tells the size.

Flashcards

Flashcard

H0: μ1 = μ2

The no-difference null for two groups

Equivalently μ1 − μ2 = 0.

Test your understanding

Question 1 of 5

Comparing two tutoring methods, the null hypothesis is:

Question 2 of 5

With s1 = 8, n1 = 25, s2 = 9, n2 = 30, the standard error of the difference is about:

Question 3 of 5

With x̄1 = 80, x̄2 = 74 and SE ≈ 2.29, the test statistic is about:

Question 4 of 5

For the difference of two INDEPENDENT means, the standard error combines the two variances by:

Question 5 of 5

The 95% interval for the difference is (1.4, 10.6) points. At α = 0.05, the two-sided test will:

← Previous topicNext topic →