Probability > Prior Probability: Uniformative, Conjugate

## What is Prior Probability?

Prior probability is a probability distribution that expresses established beliefs about an event before (i.e. prior to) new evidence is taken into account. When the new evidence is used to create a new distribution, that new distribution is called *posterior *probability. In some ways, it’s just an educated guess (i.e. an ansatz) to start off the process of finding a solution. For example, you’re on a quiz show with three doors. A car is behind one door, while the other two doors have goats. You have a 1/3 chance of winning the car. This is the **prior probability.** Your host opens door C to reveal a goat. Since doors A and B are the only candidates for the car, the probability has increased to 1/2. The prior probability of 1/3 has now been adjusted to 1/2, which is a **posterior probability.**

In order to carry our Bayesian inference, **you must have a prior probability distribution**. How you choose a prior is dependent on what type of information you’re working with. For example, if you want to predict the temperature tomorrow, a good prior distribution might be a normal distribution with this month’s mean temperature and variance.

## Uninformative Priors

An uninformative prior gives you **vague information** about probabilities. It’s usually used when you don’t have a suitable prior distribution available. However, you could *choose* to use an uninformative prior if you don’t want it to affect your results too much. The uninformative prior isn’t really “uninformative,” because any probability distribution will have *some* information. However, it will have** little impact on the posterior distribution** because it makes minimal assumptions about the model. For the temperature example, you could use a uniform distribution for your prior, with the minimum values at the record low for tomorrow and the record high for the maximum.

## Conjugate prior distribution

A conjugate prior distribution (or just *conjugate prior*) is a prior distribution which — along with the data model — produces a posterior distribution that shares the same functional form as the prior distribution, but with new and updated parameter values [1]. In other words, if the prior distribution and the posterior distribution are members of the same parametric family, then the prior is a conjugate for the likelihood function.

## References

[1] Hitchcok, D. STAT 535: Chapter 5: More Conjugate Priors. Retrieved August 20, 2023 from: https://people.stat.sc.edu/hitchcock/stat535slides5BRBhandout.pdf