Eight reasons not to test for baseline group equivalence in a parallel groups pretest-posttest study

Abstract

The parallel groups pretest-posttest design has long been prominent in quantitative research of SLA. Ideally, groups are formed by random assignment of individuals. But with or without random assignment, groups may differ substantially on key pre-treatment measures such as pretest scores. When faced with non-equivalent groups, many SLA researchers have tested the difference(s) for statistical significance in the belief that p > .05 allows a main statistical analysis which assumes that the pretreatment group means do not differ. The literature of applied statistics includes numerous accounts of why such “baseline equivalence” (BE) testing is misguided. Yet BE tests continue to be reported in SLA journals at all levels of reputation. This paper describes BE testing, reviews its flaws, shows that the practice persists, and discusses possible reasons why BE tests may be thought to be legitimate, and considers options in study planning that lead to superior results and avoid conditions that appear to make BE testing necessary.