Statistics › Section H · Confidence intervals › Topic H.1

Point estimates and good estimators

A point estimate is a single best guess for a population number. This topic builds the vocabulary of estimation - estimators, estimates, and the properties that make some guesses better than others - and shows why no point should ever travel alone.

From sample to single best guess

The last chapter described how sample statistics vary; this one puts that knowledge to work. A point estimate is a single number - computed from a sample - offered as the best guess for a population parameter: the mean lifetime of all batteries, the share of all voters who approve. The estimate is a point because it has no width, and that is exactly its weakness: a point is precise but silent about its own reliability. Estimation theory says which guesses deserve the job.

Estimators and estimates

Two words to keep apart. An estimator is the rule - the method for computing a guess from data, such as "take the sample mean". An estimate is the number that rule produces from one particular sample. \(\bar{x}\) is an estimator (a formula) until data arrive, when it becomes an estimate (a value). The standard pairs: \(\bar{x}\) estimates \(\mu\), \(\hat{p}\) estimates \(p\), \(s^2\) estimates \(\sigma^2\). Same letter families, sample versus population - the notation tracks the story.

Unbiasedness

An estimator is unbiased if its sampling distribution is centred exactly on the parameter it estimates - aim before scatter. The sample mean \(\bar{x}\) and the sample proportion \(\hat{p}\) are unbiased for \(\mu\) and \(p\): the CLT chapter showed \(\mu_{\bar{x}} = \mu\). The sample variance earns its \(n-1\) denominator here: dividing the squared deviations by \(n\) would systematically underestimate \(\sigma^2\), because deviations measured from the sample's own mean come out slightly small - \(n-1\) corrects the aim. Bias, in short, is a property of the method, not of one unlucky sample.

Efficiency

Among unbiased estimators, prefer the tightest. Efficiency compares standard errors: the estimator with the smaller SE extracts more information from the same data. For normally distributed data, the sample mean beats the sample median - both aim at \(\mu\), but the mean's scatter is smaller, so the same \(n\) buys a more precise guess. For heavily tailed data the ranking flips: one wild outlier wrecks the mean but barely moves the median, so the median's resistance wins. Efficiency is a property of the estimator AND the distribution being estimated - there is no universal champion.

Consistency

An estimator is consistent if it gets closer to the truth as the sample grows - the long-run promise that more data buys accuracy. The law of large numbers guarantees it for \(\bar{x}\): its standard error \(\sigma/\sqrt{n}\) shrinks toward zero, so the sampling distribution collapses onto \(\mu\). Consistency is the minimum requirement for a serious estimator - one that fails it does not improve with data, and no amount of sampling rescues it. Any estimator whose bias and SE both shrink to zero as \(n\) grows is consistent.

Maximum likelihood, in brief

Where do estimators come from in the first place? One general recipe, maximum likelihood, picks the parameter value under which the observed data would have been most probable. Suppose 18 of 30 independent trials succeed. The chance of exactly that result is largest when \(p = 18/30 = 0.6\), so the maximum likelihood estimate is \(\hat{p} = 0.6\): the familiar sample proportion, rediscovered. The recipe matters because it still works for complicated models with no obvious formula, which is why it runs through modern statistics and machine learning.

Worked example: battery lifetimes

A lab tests \(n = 50\) batteries, finding \(\bar{x} = 412\) hours and \(s = 35\) hours. The point estimate of the mean lifetime of all such batteries is 412 hours. How firm is it? 1. The standard error: \(\frac{s}{\sqrt{n}} = \frac{35}{\sqrt{50}} \approx 4.95\) hours. 2. Reading: about 95% of sample means land within two standard errors of \(\mu\), so the estimate is probably within about \(2 \times 4.95 \approx 10\) hours of the truth. 3. Check: \(4.95^2 \times 50 \approx 1225 = 35^2\), so the SE is right.

Worked example: a poll's point and its wobble

"Support stands at 54%", announces a poll of \(n = 500\) - and with no measure of wobble, the number invites overconfidence. 1. The estimate: \(\hat{p} = 0.54\). 2. The standard error, using the estimate for \(p\): \(\sqrt{\frac{0.54 \times 0.46}{500}} \approx 0.022\). 3. Reading: sample proportions typically land within about two standard errors of \(p\), so the truth is probably within about 4.5 percentage points of 54%. 4. Check: the point estimate "54%" now travels with its wobble - roughly 54% ± 4.5 points - which is exactly what confidence intervals will formalise in the next topic. The habit to form here: every point travels with its SE.

Common mistakes to avoid

First, using "estimator" and "estimate" interchangeably - the rule and the number are different things. Second, reporting a point estimate with no standard error. Third, assuming an unbiased estimator is close to the truth: unbiased describes the aim, and a small sample still scatters widely. Fourth, forgetting that efficiency depends on the distribution being estimated - the mean is not always the best centre. Fifth, treating consistency as optional: an estimator that ignores \(n\) never sharpens. Sixth, confusing the sample's \(s\) with the standard error - \(s\) spreads individuals, \(s/\sqrt{n}\) spreads means.

Key takeaways

An estimator is the rule; an estimate is its value from one sample: \(\bar{x}\) for \(\mu\), \(\hat{p}\) for \(p\), \(s^2\) for \(\sigma^2\). Good estimators are unbiased (centred on the parameter - the \(n-1\) in \(s^2\) buys this), efficient (small standard error for the job and the distribution), and consistent (converging as \(n\) grows, by the law of large numbers). A point estimate carries no reliability information of its own: the battery estimate 412 hours came with an SE of about 4.95, and every point should travel with its SE - which is exactly what confidence intervals formalise next.

Flashcards

Flashcard

Point estimate

A single number offered as the best guess for a parameter

Precise, but silent about reliability.

Test your understanding

Question 1 of 5

In 60 independent trials there are 21 successes. What is the maximum likelihood estimate of p?

Question 2 of 5

An estimator is unbiased when:

Question 3 of 5

Why does the sample variance divide by n − 1 rather than n?

Question 4 of 5

A sample of 50 batteries has s = 35 hours. What is the standard error of the mean?

Question 5 of 5

A consistent estimator is one that:

← Previous topicNext topic →