In hypothesis testing, the significance level (denoted by the Greek letter alpha, α) denotes the probability of committing a Type I error, i.e., rejecting the null hypothesis when the null hypothesis is true. Its complement, (1-α), gives the Confidence level, a measure associated with the confidence interval for the parameter(s), obtained from the test. The selected alpha level determines the region of rejection for the null hypothesis.
Consider a one-sided, right-tailed test of the mean from a single Normal population. If alpha is set at the 5% level, then the cut-off value for the calculated z-statistic (the test statistic) is 1.645 (the 95th percentile of the normal distribution) and the rejection region for H0 is the area to the right of this value (shown as the blue shaded area in the graphic). If the test statistic falls witin the blue area, the null hypothesis is rejected at the 5% level.
We choose the alpha level to be a suitable small value, generally 0.05, to evaluate our test. This is done before conducting the test. The null hypothesis is rejected if the p-value of the test is smaller than the specified alpha, or alternatively, if the value of the calculated test statistic falls within the rejection region bounded by the critical value (which is determined by the choice of alpha).
Significance Level (Alpha)Type I Error.