Statistics › Section G · The central limit theorem › Topic G.1

The central limit theorem for sample means

Draw one sample, get one mean; draw another, get a different one. Statistics vary - and the central limit theorem turns that nuisance into the foundation of all inference: averages are approximately normal, centred on the truth, with a spread that shrinks predictably as samples grow.

Averages have their own behaviour

Survey 40 shoppers about how much they spent and you get one average; survey 40 different shoppers and you get another. Neither is wrong, and the difference between them is not a mistake to hide: measuring it is what statistical inference is built on. The central limit theorem describes how sample averages vary, and its answer is astonishingly simple: averages are approximately normal, centred on the population mean, with a spread that shrinks predictably as samples grow - no matter what shape the raw data have.

Sampling variability

A parameter like \(\mu\) is a fixed number, but the sample mean \(\bar{x}\) is a random quantity: take two random samples from the same population and their means will differ. This sampling variability is natural - neither sample is wrong. Recall the data-and-sampling topic: two students surveying 50 households each found different recycling rates, and both were fair. The whole game of inference is measuring how much a statistic like \(\bar{x}\) can wander.

The sampling distribution

Imagine taking every possible sample of size \(n\) from the population and computing \(\bar{x}\) for each. The distribution of all those means is the sampling distribution of \(\bar{x}\) - the statistic's own histogram. It is a thought experiment made real by theory: the sampling distribution is what tells us how close a single sample's \(\bar{x}\) is likely to land to the true \(\mu\), which is exactly what confidence intervals and hypothesis tests will need.

Centre and spread: the standard error

The sampling distribution has two landmark properties. Centre: \(\mu_{\bar{x}} = \mu\) - the sample mean is an unbiased estimator, aimed at the truth. Spread: the standard deviation of \(\bar{x}\) is

\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}

called the standard error. It shrinks with the square root of the sample size: quadruple \(n\) and the standard error halves. That is a law of diminishing returns: every halving of the standard error costs four times the sample. Notation: the sampling distribution is written \(\bar{X} \sim N\left(\mu, \frac{\sigma}{\sqrt{n}}\right)\) when it is normal.

The central limit theorem

The theorem itself: for a large enough sample size, the sampling distribution of \(\bar{x}\) is approximately normal - whatever the shape of the population. Skewed raw data, lumpy data, even two-humped data: their averages pile into a bell. A common rule of thumb is \(n \ge 30\), with more needed for very skewed populations; if the population is itself normal, \(\bar{x}\) is exactly normal for any \(n\). The normal approximation for averages is what lets the standard-normal machinery of the previous chapter apply to almost any real problem.

Worked example: clinic waiting times

Waiting times at a clinic are skewed, with \(\mu = 20\) minutes and \(\sigma = 12\). For random samples of \(n = 36\) patients, what is the chance the sample mean exceeds 24 minutes? 1. Standard error: \(\sigma_{\bar{x}} = \frac{12}{\sqrt{36}} = 2\). 2. Shape: \(n = 36\) is large, so \(\bar{x}\) is approximately normal despite the skew. 3. Standardise and read the upper tail:

P(\bar{x} > 24) = P\left(Z > \frac{24-20}{2}\right) = P(Z > 2) = 0.0228

4. Check: 24 sits two standard errors above centre, and the 68-95-99.7 rule puts about 2.5% beyond that point, close to the table's 2.3%.

Worked example: bottle fills

A bottling line fills to \(\mu = 500\) ml with \(\sigma = 20\) ml, and a supervisor samples \(n = 25\) bottles at random. What is the chance their mean fill is under 494 ml? Assume individual fills are roughly normal, so the sample mean is close to normal even though \(n\) is below 30. 1. Standard error: \(\frac{20}{\sqrt{25}} = 4\) ml. 2. Standardise and read the lower tail:

P(\bar{x} < 494) = P\left(Z < \frac{494-500}{4}\right) = P(Z < -1.5) = 0.0668

3. Check: 494 is 1.5 standard errors below centre, and about 6.7% of a normal sits below -1.5, so the tail size matches. Note the contrast with the previous example: the question flipped to below, so the left tail was read instead.

Bias and variability

Two different questions about an estimator: is it aimed at the truth, and how tightly do its values cluster? The dartboard picture: bias is where the darts centre (aim), variability is their scatter. Random sampling controls bias - it aims the estimator at \(\mu\) by giving every sample a fair chance. Sample size controls variability - larger samples cluster their \(\bar{x}\) values more tightly. A big biased study is precisely aimed at the wrong answer; a small fair one is aimed right but scatters wildly. Good inference needs both.

Why this chapter matters

Everything downstream rests here. Confidence intervals are built by asking how far \(\bar{x}\) typically lands from \(\mu\) - a standard-error question. Hypothesis tests ask whether an observed \(\bar{x}\) is too far from a claimed \(\mu\) to be plausible - the same machinery, read as a tail area. The central limit theorem is the licence that lets all of it proceed with normal curves, even when the raw data are nothing like normal. Master the sampling distribution, and the inference chapters become arithmetic.

Common mistakes to avoid

First, confusing the three distributions: the population's, one sample's, and the sampling distribution of \(\bar{x}\) - three different histograms with three different jobs. Second, believing the CLT makes the data normal: it applies to averages, and raw data keep their shape. Third, forgetting the square root: the standard error is \(\sigma/\sqrt{n}\), not \(\sigma/n\). Fourth, using the CLT with tiny samples from very skewed populations. Fifth, quoting the standard error as if it described individuals - it describes means. Sixth, expecting two random samples to agree exactly.

Key takeaways

Sample means vary, and their distribution is predictable: centred at \(\mu\) (unbiased), with standard error \(\sigma/\sqrt{n}\), and approximately normal for large \(n\) by the central limit theorem - whatever the population's shape. Quadrupling the sample halves the standard error. The clinic example: \(\sigma = 12\), \(n = 36\), SE = 2, so \(P(\bar{x} > 24) = P(Z > 2) = 0.0228\). Random sampling controls bias; sample size controls variability. This sampling distribution is the foundation every confidence interval and hypothesis test stands on.

Flashcards

Flashcard

Sampling variability

Sample means differ from sample to sample

Neither sample is wrong.

Test your understanding

Question 1 of 5

A population has σ = 12. For samples of size 36, what is the standard error of the mean?

Question 2 of 5

For large samples, the central limit theorem says that:

Question 3 of 5

Clinic waits: μ = 20, σ = 12, n = 36. What is P(x̄ > 24)?

Question 4 of 5

If a study quadruples its sample size, the standard error of the mean:

Question 5 of 5

In the dartboard picture, random sampling controls ___ and sample size controls ___.

← Previous topicNext topic →