Statistics › Section N · Counting, Bayesian inference, and multiple regression › Topic N.4
Bayesian and frequentist views
Almost every method in this course has been frequentist; the last two topics were Bayesian. Both are legitimate mathematics, and they answer DIFFERENT questions - which is why a wise reader of research knows both languages, and knows which one a given number is speaking.
Two questions, both legitimate
Frequentist methods ask: IF the hypothesis were true, how surprising are these data? - answered by p-values, significance, confidence intervals. Bayesian methods ask: GIVEN these data, how probable is each hypothesis? - answered by posterior distributions and credible intervals. The questions run in opposite directions (data-given-hypothesis versus hypothesis-given-data), and neither answer can be converted into the other without the extra ingredient the Bayesian route requires: a prior. This course has taught the frequentist machinery chapter by chapter; the previous two topics built the Bayesian alternative. Both are mathematics; the difference is what question each is willing to answer.
Intervals: same look, different logic
A 95% confidence interval (chapter H) belongs to a PROCEDURE: over repeated samples, the method captures the true value 95% of the time - the interval itself carries no probability. A 95% credible interval (the last topic) holds 95% of the POSTERIOR: given the prior and this data, a 95% probability that the parameter lies within. The credible interval makes the statement people usually want - and often mistakenly attribute to confidence intervals. In practice, with weak priors and reasonable samples the two often produce similar numbers; the interpretations never merge.
Bayes factors: comparing hypotheses head-on
Instead of p-values, Bayesians can compare two hypotheses by how well each PREDICTS the data: the Bayes factor is the ratio of the two predictive probabilities. A Bayes factor of 10 means the data are ten times more likely under one hypothesis than the other - a direct statement about the hypotheses, in the odds language of the Bayes-theorem topic. And one property is quietly revolutionary: unlike a p-value, which can only fail to find evidence against a null, a Bayes factor can provide evidence FOR the null - data genuinely better predicted by no-effect than by effect. The standard significance test has no symmetric instrument; separate frequentist equivalence tests come closest.
The subjectivity objection - and the answer
The standing critique of Bayesian methods: the prior injects subjective judgement into what should be objective analysis. The honest Bayesian answer has three parts. First, the frequentist toolkit makes choices too - significance levels, one- versus two-sided tests, model forms - merely less visibly. Second, transparency: state the prior and justify it, so readers can judge it. Third, sensitivity analysis: repeat the analysis under different reasonable priors and show whether the conclusion moves. With much data, reasonable priors converge; with little data, the conclusion SHOULD depend on the prior - and a reader deserves to see how. The vice is not subjectivity; it is unstated subjectivity.
Bayesian thinking as everyday reasoning
The Bayesian update - shift belief proportionally to evidence - is not just statistics; it is the logic of good reasoning generally. Extraordinary claims require extraordinary evidence, because their prior probability is low: a single surprising study, a single dramatic anecdote, a single too-good offer should shift belief somewhat, not completely - and less so when prior evidence points the other way. The reader who swings with every fresh headline and the one who never budges are both failing the same update. Bayes' theorem formalises what careful thinkers already do.
Computation: how Bayes went practical
Simple models update by tidy formulas, as the beta prior for a proportion showed in the last topic. Realistic models rarely do: with many parameters, the posterior has no closed form, and modern Bayesian practice relies on simulation - Markov chain Monte Carlo (MCMC) methods, which generate many thousands of draws whose distribution settles onto the posterior, then summarise whatever matters from the draws. MCMC entered mainstream statistics around 1990, and as computing power grew, Bayesian methods spread through medicine, ecology and machine learning - the spam filters and navigation systems of the Bayes-theorem topic, scaled up.
Worked example: one trial, two readings
A clinical trial reports a treatment effect with \(p = 0.04\) - frequentist language - and, from a sceptical prior centred on no effect, a Bayesian posterior probability of benefit of 0.93. Read together: the frequentist result says the data would be unusual under no effect; the Bayesian result says that even starting cautious, benefit is now quite likely - but 7% doubt honestly remains. Neither number is wrong; they answer different questions about the same evidence. The full picture is both: surprise under the null, AND belief after the update - with the prior stated so the 0.93 can be judged, not just consumed.
Worked example: which school for which job?
Match the method to the setting. A drug regulator reviewing a new compound with no prior information: conventional frequentist machinery, significance thresholds and all - the setting's conventions exist for good reasons, and the prior is thin. A product team streaming daily conversion data on two designs: Bayesian updating - the A/B sentence ("96% probability B wins") is the decision-maker's language, and today's posterior becomes tomorrow's prior automatically. A researcher weighing a surprising new finding against a deep literature: Bayesian reading - the low prior keeps one paper from flipping belief. Check: none of these dismisses either school; each picks the toolkit whose question matches the job.
Common mistakes to avoid
First, reading a p-value as a posterior probability: p = 0.04 is not a 4% chance the null is true, since a p-value is computed by assuming the null is true. Second, calling Bayesian results "objective" without naming the prior - the subjectivity does not vanish; it hides. Third, dismissing either school wholesale: both answer real questions, and the literature uses both. Fourth, skipping sensitivity analysis when the prior carries weight. Fifth, confusing a large p-value with evidence FOR the null - a significance test cannot support the null; a Bayes factor can. Sixth, quoting a credible interval with the confidence interval's grammar - the direct probability statement is the whole point, and it is not interchangeable.
Key takeaways
Frequentist: how surprising are the data under the hypothesis? - p-values, significance, confidence intervals as procedure rates. Bayesian: how probable is the hypothesis given the data? - posteriors, credible intervals as direct probabilities, Bayes factors comparing predictive success, able to support a null, which a significance test cannot. The prior is the price of the direct answer: stated openly, tested by sensitivity, dominated by data when data are plentiful. Extraordinary claims need extraordinary evidence - Bayes as everyday reasoning. MCMC made complex models practical; modern practice is bilingual, choosing by the question and the setting. The trial's two readings - p = 0.04 and posterior 0.93 under a sceptical prior - are both true, and together say more than either.
Flashcards
Flashcard
How surprising are the data if H0 is true?
Test your understanding
Question 1 of 5
Which method can provide evidence in FAVOUR of the null hypothesis?
Question 2 of 5
A 95% CREDIBLE interval means:
Question 3 of 5
A trial reports p = 0.04 AND a posterior probability of benefit of 0.93 under a sceptical prior. These two numbers:
Question 4 of 5
The best response to the objection that priors are subjective is:
Question 5 of 5
A single dramatic study contradicts a large body of prior evidence. The Bayesian reading:
