< List of probability distributions < Analytic Distribution
Probability distributions can be classified as two types: analytic (or a priori) and non-analytic (synthetic). An analytic distribution has a closed form probability density function (pdf), which can be used to calculate probabilities while non-analytic probability distributions do not have closed form pdfs; probabilities are usually calculated from simulation experiments.
What is an analytic distribution?
An analytic distribution can be expressed in terms of a mathematical formula. This means that the probability can be calculated by plugging values into a “closed form” probability density function (pdf). Closed form pdfs are expressions for exact probability solutions, given with a finite amount of data [2].
Some examples of analytic probability distributions include:
- Beta distribution
- Binomial distribution
- Gamma distribution
- Normal distribution
- Poisson distribution.
Advantages of analytic probability distributions include that they are easy to understand and interpret and they allow for quick calculation of probabilities. However, some distributions have a high computational cost, and accuracy may be reduced for small samples.
Non-analytic probability distributions are calculated from an experiment, usually because there is no known probability experiment to fit the data. For example, the probability you’ll find a parking space near a football stadium on game day has no known probability distribution — but you could calculate the probabilities with a series of experiments (say, over the course of a year).
Non-analytic distributions have probability density functions (PDFs) that cannot be expressed in closed form. They include:
Methods to estimate probabilities for non-analytic probability distributions include histogram estimation, kernel density, Maximum likelihood estimation and Parzen window estimation. These methods can also be used when you draw from an analytic distribution but don’t know exactly which distribution your variable is drawn from.
Analytic distribution and the CLT
If we sum values drawn from normal distributions, the distribution of the sum is also normal. However, most other probability distributions lack this property; sums values drawn from other distributions do not usually have an analytic distribution [3].
But there is a workaround: we can take advantage of the Central Limit Theorem, which tells us if we add up n values from almost any probability distribution, the distribution of the sum converges to normal as the sample size increases. Thus, for large samples, sums of drawn values will have an analytic distribution — the normal distribution.
References
- Ainali, CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0, via Wikimedia Commons
- Hoeij1, M. Closed Form Solutions
- Think Stats Exploratory Data Analysis in Python