Linear Exponential family

< List of probability distributions < Linear exponential family

The linear exponential family is a large parametric family that includes many common distributions such as the normal distribution, the gamma distribution, and the Poisson distribution. The family is used in a wide variety of areas such as credibility theory in actuarial science, Bayesian analysis and big data applications.

Linear exponential distributions can be discrete or continuous. The discrete linear exponential (DLE) family — which includes the binomial distribution, geometric distribution, negative binomial and Poisson distributions –is also called the family of discrete power series distributions [1], because the probability mass function (PMF) of a DLE distribution can be expressed as a discrete power series.

Linear exponential family definition

The linear exponential family is conventionally defined as having the probability density function (PDF) form of [2]

Where

• x is a random variable
• ω is a set of parameter points such that ∫e^(ωx) dμ(x) < ∞ is a finite or infinite interval [3].
• ω(x) can be interpreted as a scalar product [4].
• The function q(ω) can be defined as q(ω) = Σ a(x)e^(ωx) for discrete distribution or ∫ a(x)e^(ωx) for continuous distributions. In some texts, q(ω) is denoted as f(ω)

An s-vector is a vector of parameters that define a discrete linear exponential (DLE) distribution. If x is an s-vector such as x = (x₁, x₂, … x_s), then ω is also an s-vector — in this case (ω₁, ω₂, … ω_s) [3].

Properties of the linear exponential family

Univariate members of the linear exponential family share the following properties:

• A PDF with an analytic cumulant generating function (CGF) is linear exponential in ω iff its cumulants κ satisfy κ_{r + 1} = dκ/dω [5].
• For the first two cumulants, the variance to mean ratio κ₂(ω)/κ₁(ω) = (1 + de^ω ) ^-1 holds true iff the distribution is binomial (d > 0), Poisson (d = 0), or negative binomial (d < 0) [6].
• Equality of standard deviation and mean [2].
• If the third cumulant is identical to zero, the linear exponential member is a normal distribution, under some weak regularity conditions [7]. The third cumulants is skewness, so this is saying that the normal has zero skew. Weak regularity conditions ensure that the moment generating function is well-defined and typically includes requirements that the distribution is non-degenerate with non-empty support.

Multivariate members of the linear exponential family share the following properties:

• A PDF with an analytic CGF is multivariate linear exponential in ω iff κ_{r + ei} (ω) = dκ_r (ω)/dω_i, r = 1, 2, …, s; where
  • κ_r is the r^th moment
  • ω_i is the i^th element of the vector ω
  • r = (r₁, r₂, …, r_s) with r ≥ 0, Σ r_i ≥ 1
  • e_i is the i^th unit basis vector [8]. A unit basis vector has a value of 1 in one dimension and 0 in all other dimensions.
• Component variables are mutually independent iff they are pairwise independent; subvectors are mutually independent iff the covariance of any two arbitrary subvector members in zero [6].
• A multivariate linear exponential family member is multivariate normal iff all cumulants of order 3 (skewness) are zero. The normal distribution is skew-symmetric, so all of its third-order cumulants are zero [6].
• A multivariate linear exponential family member is multiple Poisson iff the univariate marginals are Poisson and the distribution of the sum of any two random variables is also Poisson [9]. The first condition is necessary because a multiple Poisson distribution must have Poisson marginals. The second condition is sufficient because it guarantees that the distribution is closed under addition; if we add any two random variables from the distribution, the resulting random variable will be from the same distribution.
• A member of the multivariate linear exponential family is multivariate normal iff the variables are distributed normally in pairs and iff:
  • The regression of any two variables is linear
  • The sum of any two variables is univariate linear [9].

References

[1] Albert Noack. A class of random variables with discrete distributions. The Annals of Mathematical Statistics, 21(1):127–132, 1950.

[2] Wani, J.K., Patil, G.P. (1975). Characterizations of Linear Exponential Families. In: Patil, G.P., Kotz, S., Ord, J.K. (eds) A Modern Course on Statistical Distributions in Scientific Work. NATO Advanced Study Institutes Series, vol 17. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-1848-7_37

[3] Lehmann, E. L. (1959).Testing Statistical Hypotheses. John Wiley, New York.

[4] Wani, J. (1968). On the linear exponential family. Mathematical Proceedings of the Cambridge Philosophical Society, 64(2), 481-483. doi:10.1017/S0305004100043097

[5] Patil, G. (1963). A Characterization of the Exponential-Type Distribution. Biometrika, 50, 205-207.

[6] Bildikar S., Patil G.P. (1968). Multivariate exponential type distributions. Ann. Math. Statist. 39, 1316–1326.

[7] Bolger, E.M., andW. L. Harkness: Some characterizations of exponential-type distributions. Pacific J. Math.,16, (1), 1966, 5–11.

[8] Patil, G. P. (1965). On multivariate generalized power series distribution and its application to the multinomial and negative multinomial. In Classical and Contagious Discrete Distributions, Statistical Publishing Society, Calcutta and Pergamon Press, 183-194. Also in Sankhya Series A, 28, 225-237.

[9] Bildikar, S. On Characterization of Certain Exponential-Type Distributions. Sankhka: The Indian Journal of Statistics Series B Vol. 31, No. 1/2, Jun., 1969 (pp. 35-42)