Alpha levels are used in hypothesis tests. The significance level α is the probability of making the wrong decision when the null hypothesis is true.
Alpha Levels: Type I and Type II errors
In hypothesis tests, two errors are possible, Type I errors and Type II errors.
Type I error: Supporting the alternate hypothesis when the null hypothesis is true.
Type II error: Not supporting the alternate hypothesis when the alternate hypothesis is true.
In an example of a courtroom, let’s say that the null hypothesis is that a man is innocent and the alternate hypothesis is that he is guilty. if you convict an innocent man (Type I error), you support the alternate hypothesis (that he is guilty). A type II error would be letting a guilty man go free.
An alpha level is the probability of a type I error, or you reject the null hypothesis when it is true. A related term, beta, is the opposite; the probability of rejecting the alternate hypothesis when it is true.
This graph shows the rejection region to the far right.
How do I Pick an Alpha Level?
Alpha levels can be controlled by you and are related to confidence intervals. To get the alpha level, subtract your confidence interval from 1. For example, if you want to be 95 percent confident that your analysis is correct, the alpha level would be 1 – .95 = 5 percent, assuming you had a one tailed test. For two-tailed tests, divide the alpha level by 2. In this example, the two tailed alpha would be .50/2 = 2.5 percent. See: One-tailed test or two? for the difference between a one-tailed test and a two-tailed test.
Questions? Post on our FREE statistics forum. Our Stats Guy is always happy to answer your tricky questions.