> Types of Functions > Characteristic Function Contents:
What is a Characteristic Function?
The phrase “characteristic function” means two different things in math, depending on whether you’re working with probability theory or set theory:
- Probability theory: a characteristic function defines the probability distribution of a random variable.
- Set theory: a characteristic function assigns a value of 1 or 0 to tell you if an element is part of a set (or not).
These two definitions are not related, so the terminology can sometimes be confusing. But if you take note of the context (probability theory or set theory), it should be clear which definition you’re working with. In probability, a characteristic function completely defines a probability distribution. Completely defining a probability distribution involves defining a complex function (a mix of real numbers and imaginary numbers). The characteristic function φ(X) of a random variable X is:
φ(X) = E(eitX) = E(cos(tX)) + iE(sin(tX))
Where:
- t = a real number
- I = an imaginary unit
- E = the expected value.
While this equation may look challenging, some characteristic functions for common distributions, such as the chi-square distribution, normal distribution and binomial distribution, have already been defined: 
Finding others can be more challenging, but some rules have been formulated (much in the same way a set of rules for finding derivatives of functions have been found in calculus). For example, if a function is the sum of two independent random variables X and Y, then φ(X,Y) = X + Y [2].
Characteristic function vs. Fourier transforms
The characteristic function of a random variable is defined as the Fourier transform of its probability density function and can be expresses as a series of sinusoidal functions. For example, the characteristic function of the exponential distribution is defined as 1/(1 – it) While this is not a sinusoidal function, it can be expressed as a series of sinusoidal functions.
The Fourier transform is an important part of signal processing and quantum mechanics. In signal analysis, for example, it can break down signals into component frequencies. However, in probability and statistics, its unlikely you’ll deal with Fourier transforms, as most characteristic functions of popular distributions are known (as shown in the above image).
Characteristic functions in set theory
In set theory, the characteristic function of a set is a function that takes on the value 1 if an element belongs to the set and 0 otherwise.
This is a more specialized concept compared to the indicator function. The indicator function of a set can be leveraged to determine if any element is a member of this set. Conversely, the characteristic function of a set is only effective for determining the membership of elements that are already known to be a part of the set.
Characteristic Function vs. MGF
Moment generating functions (MGF) and characteristic functions are powerful functions that both describe the underlying features of a random variable [3]. While they are both defined as the Fourier transform of a random variable’s pdf, they do have some notable differences.
| Property | Moment-generating function (MGF) | Characteristic function (CF) |
|---|---|---|
| Type of function | Real-valued function. | Complex-valued function. |
| Convenient for practical applications | Yes. It can be calculated using numerical methods. | No. It cannot be calculated using numerical methods. |
| More powerful tool for theoretical purposes | No, the MGF of a random variable is not as powerful as the CF. | Yes, the CF is more powerful than the MGF because it can be used to prove a wider range of theorems. |
In general, the characteristic function is a more heavyweight tool than the MGF while the MGF is more user-friendly. The decision on which function to use depends on the specific application. For example, the characteristic function can be used to prove the central limit theorem, while the MGF can be used to quickly calculate a random variable’s expected value.
References
- Cheng-Few Lee, John Lee. (2010). Handbook of Quantitative Finance and Risk Management. Springer Science & Business Media.
- Richard H. Battin. (1999). An Introduction to the Mathematics and Methods of Astrodynamics. AIAA.
- Chen, Y. Lecture 1: Review on Probability and Statistics STAT 516: Stochastic Modeling of Scientific Data Autumn 2018