Statistics › Section N · Counting, Bayesian inference, and multiple regression › Topic N.3
Bayesian estimation of a proportion
Estimating a proportion is the classic Bayesian laboratory: a family of priors called beta makes the updating arithmetic almost trivially simple - add the data's successes and failures to the prior's, and read off updated beliefs about p.
The beta prior
For a proportion \(p\), the natural Bayesian prior family is the beta distribution, \(\text{Beta}(a, b)\), whose mean is \(\frac{a}{a+b}\) and whose two parameters act like PRIOR COUNTS: \(a\) prior successes and \(b\) prior failures. \(\text{Beta}(1, 1)\) is flat - uniform, no preference for any value of \(p\) - the standard "know nothing" choice. Bigger \(a\) and \(b\) mean a more confident prior (its belief is more tightly bunched); \(\text{Beta}(20, 20)\) says "\(p\) is near 0.5, strongly", where \(\text{Beta}(1, 1)\) says nothing.
Conjugate updating
The beta prior's magic is its conjugacy: paired with binomial data, the posterior is ALSO beta, with the counts simply added:
Observe 7 successes in 50 trials from a uniform \(\text{Beta}(1,1)\): the posterior is \(\text{Beta}(8, 44)\) - one prior success, one prior failure, plus the 7 and 43. No integrals, no software: Bayesian updating for a proportion is bookkeeping. Priors with this stay-in-the-family property are called conjugate priors, and they are why the proportion is the friendliest Bayesian example there is.
Worked example: a website's click-through rate
A new site tracks 50 visits, observing 7 clicks; the raw rate is \(7/50 = 0.14\). With a uniform \(\text{Beta}(1,1)\) prior: 1. Posterior: \(\text{Beta}(1+7, 1+43) = \text{Beta}(8, 44)\). 2. Posterior mean: \(\frac{8}{52} \approx 0.154\) - slightly pulled toward the prior mean of 0.5 compared with the raw 0.14, the prior's two pseudo-observations still whispering. 3. As data accumulate, the prior's fixed two counts fade: if clicks continued at 14%, after 5,000 visits the posterior mean (\(701/5002\)) and the raw rate would agree to three decimals. Check: the posterior mean is always \(\frac{a+k}{a+b+n}\) - a blend of prior counts and data, with the data's share growing as \(n\) grows.
Credible intervals
The Bayesian analogue of the confidence interval is the credible interval: the range containing a stated share of the POSTERIOR. Its interpretation is direct, and it is the statement people mistakenly make about confidence intervals: given the prior and the data, there is a 95% probability that \(p\) lies in a 95% credible interval. For the website's \(\text{Beta}(8, 44)\), the central 95% credible interval runs from about 0.07 to 0.26 (software computes the exact bounds): believable click rates from about 1-in-14 to 1-in-4 - honest width for 7 clicks. The confidence interval from chapter H covers the same ground with a different logic; with weak priors and decent samples the two intervals are usually close numerically, though never in meaning.
Predicting the next visitor
The posterior does more than locate \(p\) - it predicts future data. The probability that the NEXT visitor clicks is the posterior mean, about 0.154: each possible \(p\) weighted by how believable it now is. And full predictive distributions come the same way: draw many plausible \(p\)'s from the posterior, simulate the next 1,000 visitors under each, and the spread of outcomes - about 70 to 265 clicks in 95% of simulations - prices both the noise in clicks and the remaining uncertainty about \(p\) itself. This is what a forecast looks like when it admits it does not know the underlying rate.
A/B tests, the Bayesian way
Comparing two versions? Give each its own beta posterior and ask the decision question directly: what is the probability that B's rate exceeds A's? Computing it exactly is hard, but simulating it is trivial - draw a rate from each posterior, compare, repeat - and the answer lands as a decision-ready sentence: "there is a 96% probability that design B converts better." Contrast the p-value's indirect answer, which certifies surprise under a no-difference null and cannot say a probability about the difference at all. Both are legitimate; the Bayesian sentence is the one product managers repeat in meetings.
Worked example: shrinkage and the one-review restaurant
Bayesian estimates pull extreme small samples toward the prior mean - shrinkage - and it is a feature, not a bug. Score restaurants by the proportion \(p\) of their reviews that are 5-star. One whose only review is 5-star has a raw share of 1, yet with a uniform prior its posterior is \(\text{Beta}(2, 1)\), mean \(\frac{1+1}{1+1+1} = \frac{2}{3}\), solidly below a restaurant with 384 five-star reviews out of 400 (posterior mean \(385/402 \approx 0.96\)). The single rave barely registers because one observation cannot outweigh prior counts - exactly the scepticism small samples deserve. Rating systems, sports standings and recommendation engines all use this idea; it is the statistical cure for "best item: reviewed once".
Common mistakes to avoid
First, forgetting to ADD the prior counts: the posterior is \(\text{Beta}(a+k, b+n-k)\), not \(\text{Beta}(k, n-k)\) - the prior is a real participant. Second, treating a credible interval as if it were computed like a confidence interval: same look, different logic, different claims. Third, an extreme prior that overwhelms modest data - after a \(\text{Beta}(99, 1)\) prior, 20 failures in 20 trials still leave a posterior mean of \(99/120 \approx 0.83\). Fourth, reporting the posterior mean without its interval: 0.154 ± a wide credible range is a different fact from 0.154 ± nothing. Fifth, confusing the posterior predictive with the posterior: one describes future clicks, the other the underlying rate. Sixth, forgetting shrinkage goes away: with enough data every prior fades.
Key takeaways
Proportions pair with beta priors: \(\text{Beta}(a, b)\), mean \(\frac{a}{a+b}\), parameters as prior successes and failures, \(\text{Beta}(1,1)\) the flat know-nothing choice. Updating is addition: \(\text{Beta}(a, b) + (k \text{ of } n) \to \text{Beta}(a+k, b+n-k)\) - conjugacy, the property that keeps Bayesian arithmetic on one family. The site's 7-of-50: posterior \(\text{Beta}(8, 44)\), mean \(\approx 0.154\), 95% credible interval about 0.07 to 0.26, and the next-visitor prediction equals the mean. A/B tests read as "96% probability B wins"; shrinkage pulls one-review extremes toward the prior (2/3 for a single 5-star), tempering small samples until data dominate.
Flashcards
Flashcard
The proportion prior/posterior family
Test your understanding
Question 1 of 5
With a Beta(2, 2) prior and 6 successes in 10 trials, the posterior is:
Question 2 of 5
A uniform prior Beta(1, 1) meets 7 clicks in 50 visits. The posterior mean is about:
Question 3 of 5
A 95% credible interval for p means:
Question 4 of 5
A restaurant's only review is 5-star. With a uniform prior on its proportion of 5-star reviews, the posterior mean is 2/3, mainly because:
Question 5 of 5
In a Bayesian A/B test, the most decision-ready summary is:
