Sampling in Statistics > Efficiency & Efficient Estimator
What is Efficiency?
The meaning of “efficient” in statistics isn’t much different from the dictionary definition of efficient:
“…capable of producing desired results without wasting materials, time, or energy. ~ Miriam Webster.
In other words, an efficient procedure produces results that maximize your use of materials, time and energy. You’ll use less energy if you have smaller sample sizes, for example. So a procedure that can work with a smaller sample is usually more efficient than one that requires a larger sample.
Efficiency can refer to any procedure you want to optimize. For example, an efficient experimental design is one that produces your desired experimental results with the minimum amount of resources (e.g. time and money). An efficient hypothesis testing procedure also conserves resources in the same way. Generally, the most efficient hypothesis test, experimental design or estimator is going to be the one with the fewest observations.
What is Relative Efficiency?
Relative efficiency has a couple of meanings, depending on the context:
- As another name for relative precision in sampling. If you use simple random sampling to find the mean of a large population, relative precision and relative efficiency are equal. In other cases, they may not be equal.
- As a reference to a gold standard — a “best possible” procedure. It is the ratio of the potential procedure against the gold standard. for example, 75/100 indicates the prospective procedure is 75% as efficient as the theoretically best possible procedure.
What is an Efficient Estimator?
An estimator is a statistic that estimates some fact about the population. for example, the X̄(the sample mean) is an estimator for the population mean, μ. Your options for finding X̄ are limitless: you could have a sample of ten, fifty of three hundred and one. You could use different classes, ages, or heights (depending on what you are trying to estimate). You could use a simple random sampling technique, or a more complex one like stratified sampling. Out of all these possible scenarios, an efficient estimator is one that has small variances (the estimator with the smallest possible variance is also called the “best” estimator). In other words, the estimator deviates as little as possible from the “true” value you are trying to estimate.
The more formal definition of an efficient estimator is covered in the next article (What is the Cramer-Rao Lower Bound?):
An unbiased estimator is efficient if the variance of Θ equals the Cramer-Rao Lower Bound