A statistical technique used to help disprove or reject a particular conjecture about a process or population based on evidence from the sample data. This conjecture or hypothesis usually assumes a stand of status quo or “no difference”, called the null hypothesis. The statement held to be true if the null is false is called the alternative hypothesis. Rejection of the null hypothesis results in concluding in favor of the alternative hypothesis.
All hypothesis tests share these basic steps:
1. Select a significance level which will be used to decide the test. This must be done up front so as not to bias the test results. The significance level is the probability of committing a Type I error, i.e., the probability of rejecting the null hypothesis when it is in fact true. It is considered the more serious type of error, the other being Type II error, or failing to reject the null when it is fact false.
2. State the null and alternative hypotheses in terms of the population parameters. E.g. The means of the two populations under study are equal, or H0: µ1 = µ2 against the alternative that the means differ significantly, H1: µ1 ? µ2
3. Calculate the test statistic, chosen based on the hypotheses and test under consideration. This test statistic will form the decision criterion for rejecting the null hypothesis.
4. Obtain the p-value corresponding to the test statistic under the null hypothesis. This is the probability of observing a value as or more extreme than the test statistic assuming the null hypothesis is true.
5. Decide the test by either comparing the test statistic to a cut-off value prescribed by the significance level chosen in step 1, or by comparing the p-value to the significance level. Reject the null hypothesis if the test statistic falls in the rejection region defined by the cut-off value(s) or if the p-value is smaller than the significance level.
Type I Error
Minitab - Why We Don't Accept the Null Hypothesisby Keith M. Bower and James A. Colton - http://www.minitab.com/resources/Articles/WhyWeDontAccept.pdf Errors in Hypothesis Testing from Southwest Missouri State University and iSixSigma.com - http://www.isixsigma.com/offsite.asp?A=Fr&Url=http://www.psychstat.smsu.edu/introbook/sbk26m.htm