Type III Error and Type IV Error in Statistical Tests

Statistics Definitions > Type III error and Type IV Error

What is a Type III error?

A type III error is where you correctly reject the null hypothesis, but it’s rejected for the wrong reason. This compares to a Type I error (incorrectly rejecting the null hypothesis) and a Type II error (not rejecting the null when you should). Type III errors are not considered serious, as they do mean you arrive at the correct decision. They usually happen because of random chance and are a rare occurrence.

You can also think of a Type III error as giving the right answer (i.e. correctly rejecting the null) to the wrong question. Either way, you’re still arriving at the correct conclusion for the wrong reason. When we say the “wrong question”, that normally means you’ve formulated your hypotheses incorrectly. In other words, both your null and alternate hypotheses may be poorly worded or completely incorrect.

type III error
A two-tailed t-distribution. Image: ETSU.edu
Type III errors can generally be avoided by running a two-tailed test instead of a one-tailed test. A one-tailed test has a higher power if your hypothesized direction is correct. However, if your direction is wrong, the one-tailed test will return the probability of a Type III error (only you won’t realize this!). For example, let’s say you hypothesize that there is a difference between the means of two samples, and that the mean difference is lower. You test this theory by running a left-tailed test. The test returns a small p-value and you (correctly) reject the null hypothesis that the means are the same. However, unknown to you, the means are different: it’s just that one set is higher (i.e. you should have run a right-tailed test), not lower.

Type III errors aren’t limited to differences between means. The can happen in every type of statistical test (e.g., correlations, proportions, variances etc.).

What is a Type IV error?

A Type III error is directly related to a Type IV error; it’s actually a specific type of Type III error. When you correctly reject the null hypothesis, but make a mistake interpreting the results, you have committed a Type IV error. Some common reasons that Type IV errors happen include:

  • Aggregation bias (the wrong assumption that “what is true for the group is true for the individual”).
  • Running the wrong test for your data.
  • Collinearity among predictors.

References

Dodge, Y. (2008). The Concise Encyclopedia of Statistics. Springer.
Kotz, S.; et al., eds. (2006), Encyclopedia of Statistical Sciences, Wiley.
Levine, D. (2014). Even You Can Learn Statistics and Analytics: An Easy to Understand Guide to Statistics and Analytics 3rd Edition. Pearson FT Press


Comments? Need to post a correction? Please Contact Us.