What is Inverse Probability?
Inverse probability is the probability of things that are unobserved; or, more technically, the probability distribution of an unobserved variable. It’s generally considered an obsolete term.
Nowadays, the basis of inverse probability (determining the unobserved variable) is usually called inferential statistics, and the main problem of inverse probability—finding a probability distribution for an unobserved variable—is usually called Bayesian probability.
Inverse Probability, aka Bayesian Probability— What it Involves
Bayesian probability is often used when we want to calculate the likelihood of certain outcomes given a particular hypothesis. It’s a way of making logical inference problems into simple statistics problems, by looking at conditional probabilities and comparing outcomes given different hypothetical scenarios.
The Bayes Rule formula is:
The rule can be written with slightly different notation to illustrate the connection between a hypothesis and a condition:
Here P(H|D) is the likelihood that hypothesis H is true given a particular condition D, P(D|H) is the probability that the condition D is true given the hypothesis being true, and P(H) and P(D) are the probabilities of observing the hypothesis and the condition D, independently of each other.
Watch the video for a quick example of working a Bayes’ Theorem problem, or read a different example problem below:
Simple Inverse Probability Example: Applying Bayes Rule
A screening test for a particular genetic abnormality is highly effective; it gives 99% true positive results for those who carry the abnormality, and 95% true negative results to those who don’t. Only a very small percentage of the general population, 0.001%, carry this genetic abnormality.
Inverse probability and Bayes rule allows us to calculate what the likelihood is that a random someone carries the genetic abnormality, given a positive test. The genetic abnormality is the hypothesis, and the positive test is our condition. In our formula above, we’ll want to plug in the values:
P(D|H) = 0.99
P(H) = 0.00001
P(D), or the probability of a positive test, is just the sum of two terms. The first term is the probability of a positive test given the genetic abnormality times the likelihood that the abnormality exists. The second term will be the probability of a positive test given no genetic abnormality, times the likelihood of no genetic abnormality. So:
P(D) = P(D|H) P(H) + P(D|~H) P(~H).
That equals 0.99*0.00001 + 0.01*0.99999, or 0.0100098.
Plugging these into the formula above for Bayes rule, we get:
P(H|D) = [0.99*0.00001] / 0.0100098
Or, with no regard for significant digits, 0.00098903074. We see that though the test may be fairly reliable, the genetic abnormality is rare enough that even a positive test only leaves a person with only about a 0.09 percent likelihood of having the abnormality.
Next: Bayes Theorem Problems.
Olshausen, B. (2004). Bayesian Probability Theory. Retrieved December 5, 2017 from http://redwood.berkeley.edu/bruno/npb163/bayes.pdf.
Bayes Theorem, Stanford Encyclopedia of Philosophy
If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.Comments? Need to post a correction? Please post on our Facebook page.