Jackknife Estimator: Simple Definition & Overview

Statistics Definitions > Jackknife Estimator

What is a Jackknife Estimator?

The jackknife (“leave one out”) can be used to reduce bias and estimate standard errors. It is an alternative to the bootstrap method.

Like the bootstrap, the Jackknife involves resampling. The main differences are [1]:

• The bootstrap involves sampling with replacement many times, while the jackknife systematically leaves out an observation (or a small number of observations) at a time. This effectively involves sampling without replacement.
• The bootstrap tends to be more computationally intensive in practical use because it usually involves a large number of resamples.
• The jackknife’s estimated standard error is often observed to be larger (more conservative) than bootstrap resampling.

Overview of the Jackknife Procedure

The basic idea is to calculate the estimator (e.g. the sample mean) by sequentially deleting a single observation from the sample [2]. The estimator is recomputed until there are n estimates for a sample size of n.

As a simple example, let’s say you had five data points X₁, X₂, X₃, X₄, X₅.and you wanted to estimate, for example, the mean. You would calculate five different means:

• X₁, X₂, X₃, X₄.
• X₁, X₂, X₃, X₅.
• X₁, X₂, X₄, X₅.
• X₁, X₃, X₄, X₅.
• X₂, X₃, X₄, X₅.

Once you have your n estimates
The point estimate can be calculated as an average of the leave-on-out estimates:

Once you have all leave-one-estimates and their averages, you can calculate the standard error with the following formula:

In this case, the standard error tells you how much your estimate varies when each data point is omitted in turn. A larger spread within the leave-one-out estimates implies a larger standard error.

Why Use Jackknife Estimation?

Jackknife estimation is usually used when it’s difficult or impossible to get estimators using another method. For example:

• No theoretical basis is available for estimation,
• The statistic’s function is challenging to work with (e.g. a function with no closed form integral, which would make the usual method (the delta method) impossible),

For large samples, the Jackknife method is roughly equivalent to the delta method [3].

References

1. The Bootstrap and Jack knife. Retrieved November 2, 2019 from: https://www.biostat.washington.edu/sites/default/files/modules/2017_sisg_1_9_v3.pdf
2. McIntosh, A. The Jack knife Estimation Method. Retrieved November 2, 2019 from: http://people.bu.edu/aimcinto/jackknife.pdf
3. Ramachandran, K. & Tsokos, C. (2014). Mathematical Statistics with Applications in R. Elsevier.