Parameter Estimation is a branch of statistics that involves using sample data to estimate the parameters of a distribution.
Methods of Parameter Estimation
The techniques used for parameter estimation are called estimators.
Some estimators are:
- Probability Plotting: A method of finding parameter values where the data is plotted on special plotting paper and parameters are derived from the visual plot
- Rank Regression (Least Squares): A method of finding parameter values that minimizes the sum of the squares of the residuals.
- Maximum Likelihood Estimation: A method of finding parameter values that, given a set of observations, will maximize the likelihood function.
- Bayesian Estimation Methods: A family of estimation methods that tries to minimize the posterior expectation of what is called the utility function. In practice, what this means is that existing knowledge about a situation is formulated, data is gathered, and then posterior knowledge is used to update our beliefs.
Qualities of Estimators
If the value an estimator estimates for the parameter, θ’, always converges to the actual parameter value θ as the quantity of data used for parameter estimation increases, we say an estimator is consistent.
The bias of an estimator is the deviation of the expectation from the actual true value. If, for a given estimator, the bias is zero, we say that that estimator is unbiased.
A third statistic that tells us about the reliability of an estimator is the variance. If an estimator has lower variance than another we say it is more efficient, and we can calculate the efficiency of estimator p relative to estimator q as (Var(θ’p))/Var(θ’q).
Levy, Roger. Parameter Estimation. Chapter 4 in Probabilistic Models in the Study of Language, a work in progress. Retrieved from http://idiom.ucsd.edu/~rlevy/pmsl_textbook/chapters/pmsl_4.pdf on August 12, 2018.
Cadima, Emygdio. Estimation of Parameters. Chapter 7 in Fish Stock Assessment Manual. Fod and Agricultural Organization of the United Nations. Rome, 2003 Retrieved from http://www.fao.org/docrep/006/X8498E/x8498e0e.htm on August 12, 2018.
Eshky, Ashiel. Bayesian Methods of Parameter Estimation. Retrieved from http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/AV0809/eshky.pdf on August 16, 2018.