AP statistics or Elementary Statistics. You’ve got your formulas, your probabilities, and your rigid calculations. Bayesian statistics uses some of that information, but it isn’t the be-all and end all. Bayesians add prior probabilities: Has this happened before? Is it likely, based on my knowledge of the situation, that it will happen?).
There are a few technical differences about how each camp approaches statistics, but they aren’t very intuitive. Before I delve into the technical differences, take a look at the visual below. They probably tell you all you need to know:
Not quite there yet? How about this one (idea from George Casella’s paper Bayesians and Frequentists):
Now Onto the More Technical Stuff
Frequentist statistics uses rigid frameworks, the type of frameworks that you learn in basic statistics, like:
For every statistics problem, there’s data. And for every data set there’s a test (and every test has its own rigid rules). The tests are based on the fact that every experiment can be repeated infinitely. Deviation from this set of rules is never allowed, and if you dare to deviate, your methods will be chided as statistically unsound.
Bayesian statistics is arguably more intuitive and easier to understand. Take a typical conclusion from a hypothesis test.
A frequentist might say: “If H0 is true, then we would expect to get a result as extreme as the one obtained from our sample 2.9% of the time. Since that p-value is smaller than our alpha level of 5%, we reject the null hypothesis in favor of the alternate hypothesis.”
On the other hand, a Bayesian might say that the odds of H0–the sun exploding tonight– are about a billion to one.
Which do you find more intuitive?
Casella, G. (undated) “Bayesians and Frequentists.” ACCP 37th Annual Meeting, Philadelphia, PA