Alpha levels are used in hypothesis tests. The significance level α is the probability of making the wrong decision when the null hypothesis is true.
Type I and Type II errors
In hypothesis tests, two errors are possible, Type 1 and Type II errors.
Type 1 error: Supporting the alternate hypothesis when the null hypothesis is true.
Type 2 error: Not supporting the alternate hypothesis when the alternate hypothesis is true.
In an example of a courtroom, let’s say that the null hypothesis is that a man is innocent and the alternate hypothesis is that he is guilty. if you convict an innocent man (Type 1 error), you support the alternate hypothesis (that he is guilty). A type II error would be letting a guilty man go free.
An alpha level is the probability of a type I error, or rejecting the null hypothesis when it is true. A related term, beta, is the opposite; the probability of rejecting the alternate hypothesis when it is true.
This graph shows the rejection region to the far right.
Alpha levels can be controlled by the researcher and are related to confidence levels. For example, if you want to be 95 percent confident that your analysis is correct, the alpha level would be 1 – .95 = 5 percent, assuming you had a one tailed test. For two-tailed tests, divide the alpha level by 2. In this example, the two tailed alpha would be .50/2 = 2.5 percent.