What is the Delta Method?
The delta method is a way to approximate random variables along with their covariances, means, and variances. The method can also calculate standard errors for complicated statistical estimates. Generally speaking, it is very similar to the Central Limit Theorem.
Other names for the delta method, which is identical to the infinitesimal jackknife (Jaeckel, 1992) include:
- Method of statistical differentials,
- Propagation of errors formula,
- Taylor series method.
Delta Method and The Taylor Series
A Taylor series approximation (from calculus) forms the underlying basis for the delta method. The method uses the Taylor series expansion of the regression’s inverse link function; The result is used as to derive variation around a point. The hand calculations are a bit cumbersome; They involve some intermediate calculus, including finding higher order terms for the Taylor series (which is not too difficult, but usually requires a bit of practice and the assistance of symbolic math software). Thus, exactly how the method works is beyond the scope of this article. But if you’re interested in exactly how the Taylor series is used, see Alex Papanicolaou’s Taylor Approximation and the Delta Method.
Disadvantages
If your original parameter estimates aren’t normally distributed, the delta method won’t work well; The method will underestimate standard errors (i.e. will result in a downward bias); In some cases, the underestimate may be significantly incorrect (LePage & Billard, 1992).
The bootstrap method is generally more reliable, especially when the data is non normal. For small samples, the t-distribution may provide better estimates (Alho & Spencer, 2005).
References
Alho, J. & Spencer, B. (2005). Statistical Demography and Forecasting. Springer Science and Business Media.
Jaeckel, L. (1992). The infinitesimal jackknife. Memorandum MM72-1215-11, Bell Labs, Murray Hill, NJ.
LePage, R. & Billard, L., (1992). Exploring the Limits of Bootstrap. John Wiley & Sons.
Papanicolaou, A. Taylor Approximation and the Δ method.