Familywise Error Rates > Holm-Bonferroni Method

You may want to read this article first: Familywise Error Rates.

## What is the Holm-Bonferroni Method?

The **Holm-Bonferroni Method **(also called Holm’s Sequential Bonferroni Procedure) is a way to deal with familywise error rates (FWER) for multiple hypothesis tests. It is a modification of the **Bonferroni correction**. The Bonferroni correction reduces the possibility of getting a statistically significant result (i.e. a Type I error) when performing multiple tests. Although the Bonferroni is simple to calculate, it suffers from a lack of statistical power. The Holm-Bonferroni method is also fairly simple to calculate, but it is more powerful than the single-step Bonferroni.

## Formula

The formula to calculate the Holm-Bonferroni is:

**Where**:

- Target alpha level = overall alpha level (usually .05),
- n = number of tests.

This next example shows how the formula works.

## Example

**Question**: Use the Holm-Bonferroni method to test the following four hypotheses and their associated p-values at an alpha level of .05:

- H
_{1}= 0.01. - H
_{2}= 0.04 - H
_{3}= 0.03 - H
_{4}= 0.005

**Note**: we already know the p-values associated with each hypothesis. If you don’t know the p-values, run a test for each hypothesis before attempting to adjust FWER using the Holm-Bonferroni method.

Step 1: **Order the p-values from smallest to greatest:**

- H
_{4}= 0.005 - H
_{1}= 0.01 - H
_{3}= 0.03 - H
_{2}= 0.04

Step 2: **Work the Holm-Bonferroni formula for the first rank:**

HB = Target α / (n – rank + 1)

HB = .05 / 4 – 1 + 1 = .05 / 4 = .0125.

Step 3: **Compare the first-ranked (smallest) p-value from Step 1 to the alpha level calculated in Step 2:**

*Smallest p-value, in Step 1 (H _{4} = 0.005) < Alpha level in Step 2 (.125).*

If the p-value is smaller, reject the null hypothesis for this individual test.

The p-value of .005 *is* less than .125, so the null hypothesis for H_{4} is rejected.

Step 4: **Repeat the HB formula for the second rank **.

HB = Target α / (n – rank + 1)

HB = .05 / 4 – 2 + 1 = .05 / 3 = .0167

Step 5: **Compare the result from the HB formula in Step 4 to the second-ranked p-value: **

*Second ranked p-value, in Step 1 (H _{1} = 0.01) < Alpha level in Step 2 (.0167).*

The p-value of .01 is less than .0167, so the null hypothesis for H

_{1}is rejected as well.

Step 6: **Repeat the HB formula for the third rank**.

HB = Target α / (n – rank + 1)

HB = .05 / 4 – 3 + 1 = .05 / 2 = .025

Step 7: **Compare the result from the HB formula in Step 6 to the third-ranked p-value: **

*Third ranked p-value, in Step 1 (H _{3} = 0.03) > Alpha level in Step 6 (.025).*

The p-value of .03 is greater than .025, so the null hypothesis for H

_{3}is not rejected.

The testing stops when you reach the first non-rejected hypothesis. All subsequent hypotheses are non-significant (i.e. not rejected).

**Reference**:

Holm, S. 1979. A simple sequential rejective multiple test procedure. Scandinavian Journal of Statistics 6:65-70

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments are now closed for this post. Need help or want to post a correction? Please post a comment on our Facebook page and I'll do my best to help!
Hello! There seem to be very few resources on Holm-Bonferroni correction, so this has been a great help.

I am confused about why the “Target p-value” is equal to .05. Is this actually the target alpha?

Thanks for your comment and I’m glad you found it helpful. Yes, it’s the target alpha (I changed it).

Regards,

S