Beta-Binomial Distribution: Definition

Probability Distributions > Beta-Binomial Distribution

What is the beta-binomial distribution?

 

The beta-binomial distribution is a discrete probability distribution that uses the beta distribution as a prior distribution for the probability of success in a binomial experiment. While the binomial distribution has fixed probabilities, the beta-binomial probabilities can vary from trial to trial, which makes it a more flexible distribution than the binomial.

This simple Bayesian model has been used for decades to make informed predictions in fields such as cognitive science, epidemiology, intelligence testing and marketing.

Properties of the beta binomial distribution

The probability mass function (pmf) for the beta-binomial is:  

Where x ∈ { 0, …, n } and B = the beta function. Two shape parameters α > 0 and β > 0 define the probability of success.

• For large values of α and β the distribution approaches a binomial distribution. In other words, the binomial distribution is the limiting distribution.
• When α and β are both equal to 1, the distribution is a discrete uniform distribution from 0 to n. This is because when the beta function has α = 1 and β = 1, it simply equals 1, meaning that probability of getting any number of successes is equal.
• When n = 1, the distribution is equal to a Bernoulli distribution (which models a single trial) with p chosen from a beta distribution, which has mean α/(α + β).

The mean of the beta-binomial distribution is nα / (α + β).

The variance is the product of two terms [2]:

• The first term, nαβ / (α + β)² , is the variance for a binomial distribution with the same expected value as the beta-binomial distribution.
• The second term, (α + β + n) / (α + β + 1), is a multiplier greater than 1 for n > 1.

This means that a beta-binomial distribution with n > 1 always has a larger variance than a binomial distribution with the same expected value and number of trials.

One major difference between a binomial and beta-binomial is that in a binomial distribution, p is fixed for a set number of trials; in a beta-binomial, p is not fixed and changes from trial to trial.

Deriving the Formula

Let’s say you have m items on an test, and each item is tested n times. The binomial distribution formula is: P(X=x_i) = _nC_xi * P^xi_i * (1 – p_i)^{n – x_i}, i =1,2,3,…,n 

Where:

• P = binomial probability,
• x_i = total number of “successes” (pass or fail, heads or tails etc.) for the ith trial,
• p_i = probability of a success on an individual trial,
• n = number of trials,

Perhaps the simplest way to think of the beta-binomial is to take the above formula and think of p as being randomly drawn from a beta distribution. However, to write the formula out in full, take the probability density function for the Beta Distribution:
…and combine it with the binomial distribution formula to get a joint pdf.
Which can also be written (using Beta distribution properties) as:

Difference between the binomial and beta-binomial distribution

The major difference between a binomial and beta-binomial is that in a binomial distribution, p is fixed for a set number of trials; in a beta-binomial, p is not fixed and changes from trial to trial. One benefit is that the beta-binomial can be used to model data that is overdispersed, which means that the variance is greater than the mean. This can happen in a many situations, such as when there are more extreme values in a dataset than would be expected if the data followed a normal distribution.

References

1. Nschuma, CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0, via Wikimedia Commons
2. Beta-binomial (n, α, β) distribution. Retrieved July 14, 2023 from: https://www.acsu.buffalo.edu/~adamcunn/probability/betabinomial.html