Statistics How To

Bayesian Estimator: Definition

Bayesian Statistics >

A bayesian estimator is an estimator of an unknown parameter θ that minimizes the expected loss for all observations x of X.

In other words, it’s a term that estimates your unknown parameter in a way that you lose the least amount of accuracy (as compared with having used the true value of that parameter).

Bayes Estimators & The Loss Function

A bayesian estimator is a function of observable random variables, variables you observed in the process of your research. Call your observable variables X1 X2…Xn

Your data may be able to be represented by the function f(x|θ), where θ is a prior distribution. However, you don’t know the actual value of θ, so you have to estimate it. An estimator of θ is a real valued function δ(X1 . . . Xn).

The loss function L(θ, a) where a ε R, is also a real valued function of θ. Our estimate here is a, and L(θ, a) tells us how much we lose by using a as an estimate when the true, real value of a parameter is θ.

There are different possible loss functions. For instance, the squared error loss function is given by L(θ a) = (θ –a)2. The absolute error loss function would be L(θ a) = |θ –a|.

In the first definition of Bayesian estimator at the beginning of this page, we said it was an estimator that minimized expected loss. That loss is represented by a loss function like one of those we’ve just described.

We can find the minimum expected loss by integrating. For a given X = x, the expected loss (E) is:



In this formula the Ω is the range over which θ is defined. p(θ | x) is the likelihood function; the prior distribution for the parameter θ over observations x. Call a*(x) the point where we reach the minimum expected loss. Then, for a*(x) = δ*(x), δ*(x) is the Bayesian estimate of θ.

Sources

Brynjarsdottir, Jenny. STAT 611 Lecture Notes: Lecture 12, Estimation.
Retrieved from https://www2.stat.duke.edu/courses/Fall12/sta611/Lecture12.pdf on March 9, 2018
Shiryaev, A. N. Bayesian estimator, Encyclopedia of Mathematics. Retrieved from : http://www.encyclopediaofmath.org/index.php?title=Bayesian_estimator&oldid=19043 on March 4, 2018.
Zhu, Wei. Bayesian Inference for the Normal Decision. Retrieved from http://www.ams.sunysb.edu/~zhu/ams571/Bayesian_Normal_wide.pdf on March 4, 2018

------------------------------------------------------------------------------

Need help with a homework or test question? Chegg offers 30 minutes of free tutoring, so you can try them out before committing to a subscription. Click here for more details.

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments? Need to post a correction? Please post on our Facebook page.
Bayesian Estimator: Definition was last modified: March 17th, 2018 by Stephanie