← Back to Statistics

Sampling: how data are collected

Good conclusions need good data. How a sample is chosen matters more than how large it is.

Why sample?

Measuring a whole population is usually impossible, too expensive, or destructive (you cannot crash-test every car). A well-chosen sample of a few thousand can represent a population of millions remarkably well. A badly chosen sample of millions can be useless.

Simple random samples

In a simple random sample (SRS), every group of \(n\) individuals has the same chance of being chosen. This removes the researcher's influence over who is selected and allows the use of probability to measure uncertainty. Random number generators or shuffled lists produce SRSs.

Other random designs

A stratified sample divides the population into groups (strata), such as age bands, and takes an SRS within each, guaranteeing every group is represented. A cluster sample randomly selects whole groups, such as schools, and surveys everyone in them, which is cheaper but less precise. Systematic samples take every \(k\)th individual from a list.

Bias in sampling

Bias means a method systematically favours some outcomes. Convenience samples (people nearby) and voluntary response samples (people who choose to answer, like online polls) are biased. Undercoverage leaves groups out, nonresponse occurs when chosen people do not answer, and response bias arises from leading questions or social pressure.

A famous failure

In 1936, the Literary Digest polled over 2 million people and predicted a landslide defeat for Franklin Roosevelt. Its sample came from telephone and car owners, wealthier than average during the Depression, and only people who mailed back responses. Roosevelt won easily. Size could not fix a biased method.

Observational studies and experiments

An observational study measures variables without intervening; an experiment deliberately imposes a treatment. Only well-designed experiments, with random assignment of treatments, can show cause and effect. Observational studies can show association but are vulnerable to confounding.

Principles of experimental design

Control: compare treatment with a control group. Randomise: assign subjects to groups by chance to balance unknown factors. Replicate: use enough subjects to see real effects. Blinding, where subjects (and ideally evaluators) do not know who got which treatment, prevents expectations from biasing results; a placebo helps.

Common mistakes

Believing a huge sample guarantees accuracy. Treating voluntary online polls as representative. Inferring causation from observational data. Confusing random sampling (who is in the study) with random assignment (who gets which treatment).

Key takeaways

Random sampling avoids selection bias and lets us quantify uncertainty. Stratified and cluster samples are useful variations. Watch for convenience, voluntary response, undercoverage, nonresponse, and response bias. Experiments with random assignment establish causation; observational studies show association.

Flashcard

Simple random sample

Every group of n has the same chance

The gold standard for sampling.

Check your understanding

A website asks visitors to vote on a policy, and 80% of 50,000 voters oppose it. The main problem is:

← Previous lesson Next lesson →