< Probability and statistics definitions < Convolution of probability distributions
What is convolution in probability?
Convolution in probability is a way to find the distribution of the sum of two independent random variables, X + Y. It is similar to the general meaning in math: in fact, convolution in probability and convolution in math both refer to the same mathematical operation.
- In math, convolution (f * g) sums the products of two functions f and g.
- In probability, convolution (F * G) sums the products of two probability distributions F and G [1].
The convolution operation (*) is both associative and commutative.
Finding convolutions
The calculation of convolutions can sometimes be performed analytically; for example, the sum of two normally distributed random variables is also normally distributed, with the new distribution having a mean and variance that are the sums of the two original distributions. In other cases, convolutions may need to be determined numerically — usually with software [2]. This is partly because some convolutions are hard to find, and partly because calculating the sums of random variables can be a tedious and lengthy process.
Suppose we have two independent random variables X and Y characterized by their respective density (or mass) functions fX and fY. If we add their values, the convolution gives us the probability distribution of the resulting random variable X + Y.
For discrete random variables, we can find Z = X + Y with the formula [3]
For continuous random variables, the formula is
The subscript x∈ΩX in the formula means that the formula is valid for x-values of x that are in the sample space ΩX . The sample space is the set of all possible values that the random variable X can take on.
Convolution and “sums of random variables”
You may have come across other “sums of random variables” in probability, but they are not all defined as convolutions — a term which is reserved for sums of PMFs or PDFs. For example, sums of random variables for characteristic functions and moment generating functions (MGFs) are the element-wise product. Element-wise refers to the act of performing an operation on each element of a matrix or vector. For example, if we have a vector of numbers, we could add each element of the vector to 10.
The cumulative distribution function (CDF) of the sum of two independent random variables is also a convolution — of the CDFs of the individual random variables.
References
[1] Rolski, T. et al. (!999). Stochastic Processes for Insurance and Finance. p. 28.
[2] Convolution of two probability distributions. Retrieved August 21, 2023 from: https://search.r-project.org/CRAN/refmans/bayesmeta/html/convolve.html
[3] Tsun, A. Chapter 5. Multiple Random Variables 5.5. Retrieved August 21, 2023 from: https://courses.cs.washington.edu/courses/cse312/20su/files/student_drive/5.5.pdf