This pie chart shows a distribution with different categories [1].The term “categorical distribution” has come to mean two separate things:
Informally, it’s any distribution with categories;
Alternatively (and more precisely), it’s a generalization of the Bernoulli distribution for a categorical random variable.
While a random variable in a Bernoulli distribution has two possible outcomes, a categorical random variable has multiple possibilities. The sample space for a Bernoulli distribution is {0, 1} and for a categorical distribution, it’s {0,1…n}. For example, a die roll, where there are six outcomes {1, 2, 3, 4, 5, 6} is a categorical distribution. When there is a single trial, the categorical distribution is equal to a multinomial distribution. To confound the terminology confusion, some fields — such as machine learning — use the term categorical distribution as a synonym for a multinomial distribution when “categorical distribution” would be the more precise term [2].
As this distribution only deals with discrete outcomes, it is also sometimes called a discrete categorical distribution or even a discrete distribution. However, a discrete distribution refers to a general class of distributions and not a specific family.
Examples of a categorical distribution
A categorical distribution must have K ≥ 2 potential outcomes and n = 1 trial. There are many examples of this distribution in real life, including:
Throw a six-sided dice fifty times and observe the outcomes. The possible outcomes (the sample space) are 1, 2, 3, 4, 5, 6. Each outcome has a probability of 1/6. The number of trials, “n” is 10.
Count the number of times a word appears in a book. The possible outcomes are any words in the text (e.g. the, and, to…). The probability depend on the number of words in the text.
Count the number of times a player on a hockey team scores a goal. The possible number of outcomes (i.e. goals) depend on the length of the game but will likely be somewhere between 0 and 10. Different probabilities will be assigned to the players, depending on their positions and their ability. For example, a forward has a higher chance of scoring a goal than a goalkeeper. And a forward with an excellent scoring record has a higher probability than a player with a poor record.
