Statistics How To

Normalizing Constant: Definition

Share on

Statistics Definitions >


A normalizing constant ensures that a probability density function has a probability of 1. The constant can take on various guises: it could be a scalar value, an equation, or even a function. As such, there isn’t a “one size fits all” constant; every probability distribution that doesn’t sum to 1 is going to have an individual normalization constant.

In many cases, the nonnormalized density function is known, but the normalization constant is not. In simple PDFs, the constant may be calculated relatively easy. In other cases, especially when the constant is a complicated function of the parameter space, it may be impossible to calculate.

Normalization Constant Examples

In a beta distribution, the function in the denominator of the pdf acts as a normalizing constant.

normalizing constant

The normalizing constant for the Von Mises Fisher distribution.



The normalizing constant in the Von Mises Fisher distribution is found by integrating polar coordinates.

In the Kent distribution, the normalizing constant is an equation:
another nc

For Bayes’ rule, written as [1]:
bayes rule normalisation constant

The denominator is the normalizing constant ensuring the posterior distribution adds up to 1; it can be calculated by summing the numerator over all possible values of the random variable. Calculation of the normalization becomes much more difficult for procedures like Bayes factor model selection or Bayesian model averaging; these are notoriously difficult to calculate and usually involve high dimensional integrals that are impossible to solve analytically [2].

References

[1] Murphy, K. In Symbols. Retrieved November 16, 2021 from: https://www.cs.ubc.ca/~murphyk/Bayes/bayesrule.html
[2] Gronau, Q. et al. bridgesampling: An R Package for Estimating Normalizing Constants. Retrieved November 16, 2021 from: https://cran.r-project.org/web/packages/bridgesampling/vignettes/bridgesampling_paper.pdf

CITE THIS AS:
Stephanie Glen. "Normalizing Constant: Definition" From StatisticsHowTo.com: Elementary Statistics for the rest of us! https://www.statisticshowto.com/normalizing-constant-definition/
------------------------------------------------------------------------------

Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!

Comments? Need to post a correction? Please post a comment on our Facebook page.