Statistics › Section H · Confidence intervals › Topic H.6
From intervals to decisions
An interval of 0.4 to 3.8 points says more than "B won": it says how big the win could be and how small. Learning to read the bounds against the decision is what turns an interval into action.
Intervals are decision tools
A confidence interval is usually reported as a result; this topic treats it as an instrument for deciding. What counts as plausible? Is the effect big enough to act on? Is the evidence sharp enough to commit? The answers live not in the interval's centre but in its bounds - and the discipline of reading bounds, precision and thresholds is what separates statistics that inform from statistics that decorate.
Plausible values
Every value inside a 95% interval is a plausible value for the parameter: the data are consistent with it. Every value outside has been argued against at the usual strength. An interval therefore answers many questions at once - "could \(\mu\) be 10? Could it be 0?" - where a single test answers one. The formal version of this reading arrives with hypothesis tests in the next chapter: for a mean, a two-sided test at the 0.05 level rejects exactly the values outside the 95% interval (for a proportion the match is close but not exact, because the test's SE uses the null value). Two tools, one body of evidence.
Precision and decisions
Suppose a new medicine's interval for blood-pressure reduction runs from 1 to 15 mmHg. The effect is statistically significant - zero is outside the interval - but is it tiny or substantial? The interval cannot say, and a decision that hinges on size, such as whether the side effects are worth it, cannot be made. A larger study might narrow the interval to 7 to 10 mmHg: now the effect is not merely real but usefully large, and the decision follows. Significance says an effect exists; precision says what it is worth. Decisions run on the second.
One-sided intervals
Sometimes only one bound matters. A food regulator does not care how low a contaminant's mean level might be, only that it stays below the safety limit: a one-sided upper confidence bound - "we are 95% confident the mean is below the limit" - answers exactly the question asked, and its bound sits closer to the estimate than the two-sided upper limit (\(z^* = 1.645\) instead of 1.96 at 95%) because all 5% of the error is placed in one tail. The same logic fits a manufacturer who must be confident a cable's minimum strength exceeds a rating: a one-sided lower bound. Match the interval's shape to the decision's shape.
Worked example: the A/B test
An online store tests two checkout designs at random. Design B's purchase rate runs 2.1 percentage points higher, with a 95% interval of 0.4 to 3.8 points. 1. The interval excludes 0: the two designs very likely differ - 0 is not a plausible value for the difference. 2. The lower bound, 0.4 points, is the smallest plausible gain: even in the least favourable supported case, B is slightly ahead. 3. The upper bound, 3.8, is the most B's promoters can honestly claim. 4. Decision: switch - but plan around the lower bound, 0.4 points, because the business must survive the least favourable plausible truth.
Worked example: a decision the interval cannot make
A district considers a reading program whose 95% interval for improvement runs from 2 to 9 points; the program pays for itself only above 3 points. 1. The interval spans the threshold: 2, 3 and 9 are all plausible. 2. So the evidence supports "it helps" (0 is excluded) but cannot support "it pays" - the decision-relevant question. 3. Options: run a larger pilot to narrow the interval, or decide on other grounds. 4. Check: this is the honest outcome. A wide interval is not a failure of statistics - it is the statement that the data have not yet earned the decision.
Planning: power and precision
The sample-size topic planned for a margin; decisions add the mirror concern. Under-powered studies - too small to detect the effect that matters - waste resources and manufacture failures that feed replication problems: a real effect goes unreported because the interval was born too wide. Over-sized studies carry the opposite cost: in medicine, exposing more patients than necessary to an inferior treatment is an ethics problem, not just a budget one. Planning is where good studies are made: choose the precision or the power the decision needs, then buy exactly that much.
Communicating uncertainty
An interval earns nothing sitting in a table; it must be said well. Plain language first: "Our best estimate is 27%, and the true value is likely between 24% and 30%." Graphics second: error bars communicate instantly, but label what they show - bars sometimes mark standard errors, sometimes confidence intervals, and a reader cannot tell the difference by looking. The habit to carry into every report: no estimate without its uncertainty, and no uncertainty without its meaning.
Uncertainty beyond sampling
A confidence interval prices sampling error - the chance wobble of a random sample. It does not price measurement error, a mis-specified model, or biased data collection, all of which can be far larger. Good practice pairs the interval with judgement: what could the design have missed, what did the instrument mismeasure, what does the model assume wrongly? Reporting limitations honestly is not humility theatre - it tells the reader exactly what the numbers cannot rule out.
Common mistakes to avoid
First, treating overlapping error bars as proof of no difference - overlap means the question needs a direct test or an interval on the difference, not that the answer is zero. Second, reading only the centre and never the width: 2.1 points with interval 0.4 to 3.8 is a different fact from 2.1 with 2.0 to 2.2. Third, skipping the planning step and hoping for the best. Fourth, presenting estimates bare, with no uncertainty attached. Fifth, asking a two-sided interval to answer a one-sided question. Sixth, forgetting everything an interval does not cover - bias, measurement error, model error.
Key takeaways
Inside a 95% interval: plausible values; outside: values the data argue against - the reading that hypothesis tests formalise in the next chapter. Decisions need precision, not just significance: a 1-to-15 mmHg medicine very likely works, but only the 7-to-10 version can be prescribed on. One-sided bounds match one-sided questions. The A/B test: interval 0.4 to 3.8, plan around the 0.4. When an interval spans the decision threshold, the study has not yet earned the decision - so plan the size that would. Communicate intervals plainly, label every error bar, and remember what sampling error does not cover.
Flashcards
Flashcard
Values inside the confidence interval
Test your understanding
Question 1 of 5
A tutoring scheme's 95% interval for improvement is 2 to 9 points, and it pays for itself only above 3 points. What follows?
Question 2 of 5
Why does a narrow interval (7 to 10 mmHg) serve a decision better than a wide one (1 to 15)?
Question 3 of 5
Two groups' 95% error bars overlap on a chart. This proves:
Question 4 of 5
A regulator must be confident a contaminant's mean level is below a safety limit. The interval that answers directly:
Question 5 of 5
Design B's gain has a 95% interval of 0.4 to 3.8 points. What should the business plan around?
