Comparison to Bootstrap
Like the bootstrap, the Jackknife involves resampling. The main differences are:
- The bootstrap involves sampling with replacement, while the jackknife involves sampling without replacement.
- The bootstrap tends to be more computationally intensive.
- The jackknife’s estimated standard error tends to be larger than bootstrap resampling.
Overview of the Jackknife Procedure
The basic idea is to calculate the estimator (e.g. the sample mean) by sequentially deleting a single observation from the sample. The estimator is recomputed until there are n estimates for a sample size of n. As a simple example, let’s say you had five data points X1, X2, X3, X4, X5. You would calculate the estimator five times, for:
- X1, X2, X3, X4, X5.
- X2, X3, X4, X5.
- X3, X4, X5.
- X4, X5.
Why Use Jackknife Estimation?
Jackknife estimation is usually used when it’s difficult or impossible to get estimators using another method. For example:
- No theoretical basis is available for estimation,
- The statistics’s function is challenging to work with (e.g. a function with no closed form integral, which would make the usual method (the delta method) impossible),
For large samples, the Jackknife method is roughly equivalent to the delta method.
The Bootstrap and Jack knife. Retrieved November 2, 2019 from: https://www.biostat.washington.edu/sites/default/files/modules/2017_sisg_1_9_v3.pdf
McIntosh, A. The Jack knife Estimation Method. Retrieved November 2, 2019 from: http://people.bu.edu/aimcinto/jackknife.pdf
Ramachandran, K. & Tsokos, C. (2014). Mathematical Statistics with Applications in R. Elsevier.