Preliminary Testing: The Devil of Statistics?

Abstract

In quantitative research, the selection of the most appropriate statistical test for the comparison of two-independent samples can be problematic. There is a lack of consensus in the Statistics community regarding the appropriate approach; particularly towards assessing assumptions of normality and equal variances. The lack of clarity in the appropriate strategy affects the reproducibility of results. Statistical packages performing different tests under the same name, only adds to this issue. The process of preliminary testing assumptions of a test using the sample data, before performing a test conditional upon the preliminary test, is performed by some researchers; this practice is often criticised in the literature. Preliminary testing is typically performed at the arbitrary 5% significance level. In this paper this process is reviewed, and additional results are given using simulation, examining a procedure with normality and equal variance preliminary tests.