Sampling distributions and the central limit theorem
Every sample gives a slightly different answer. The sampling distribution describes that variation, and the central limit theorem makes it predictable.
Statistics vary from sample to sample
A parameter is a fixed number describing a population, such as the true mean \(\mu\). A statistic is computed from a sample, such as \(\bar{x}\), and changes from sample to sample. This sampling variability is natural, and understanding it is the heart of statistical inference.
The sampling distribution
Imagine taking every possible sample of size \(n\) and computing \(\bar{x}\) for each. The distribution of those values is the sampling distribution of \(\bar{x}\). Its centre and spread tell us how close a single sample's estimate is likely to be to the truth.
Mean and standard error
The sampling distribution of \(\bar{x}\) is centred at \(\mu\): the sample mean is an unbiased estimator. Its standard deviation, called the standard error, is \(\frac{\sigma}{\sqrt{n}}\). Quadrupling the sample size halves the standard error, a law of diminishing returns in data collection.
\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}The central limit theorem
For a large enough sample size, the sampling distribution of \(\bar{x}\) is approximately normal, whatever the shape of the population. A common rule of thumb is \(n \ge 30\), though more is needed for very skewed populations. This theorem is why normal-based methods work so widely.
Worked example
Waiting times at a clinic are skewed with mean 20 minutes and standard deviation 12. For random samples of 36 patients, \(\bar{x}\) is approximately normal with mean 20 and standard error \(\frac{12}{\sqrt{36}} = 2\). The chance a sample averages over 24 minutes is about \(P(Z > 2) \approx 0.025\).
Bias and variability
A good estimator has low bias (centred on the truth) and low variability (tightly clustered). Think of a dartboard: bias is aiming at the wrong spot; variability is scatter. Random sampling controls bias; larger samples reduce variability.
Sampling distribution of a proportion
Similarly, \(\hat{p}\) is centred at \(p\) with standard error \(\sqrt{\frac{p(1-p)}{n}}\), and is approximately normal when \(np\) and \(n(1-p)\) are at least 10. These two sampling distributions, for means and proportions, power most of introductory inference.
Common mistakes
Confusing the population distribution, the distribution of one sample, and the sampling distribution. Thinking the central limit theorem makes individual data normal; it is about averages. Forgetting the square root of \(n\) in the standard error.
Key takeaways
Statistics vary from sample to sample; their sampling distribution describes how. \(\bar{x}\) is centred at \(\mu\) with standard error \(\sigma/\sqrt{n}\). The central limit theorem makes averages approximately normal for large \(n\). Random sampling controls bias, and larger samples reduce variability.
Flashcard
A fixed number describing a population
Check your understanding
A population has SD 20. What is the standard error of the mean for samples of size 100?
