How Well Do Simulation Studies Inform Decisions About Multistage Testing?

Abstract

Most research on multistage testing (MST) uses simulated data. This study adds to the literature by using both operational test data and simulated data to compare two different MST designs with regard to proficiency estimation accuracy and module exposure rates and by investigating whether simulation studies and operational test studies yield similar results. Two MST designs (1-2 and 1-3-4 designs) from one state's sixth-grade summative mathematics assessment across two years were compared in this study. Both simulation and operational test studies demonstrate similar results: the two MST designs yield no significant performance differences with regard to estimation accuracy and module exposure. These results provide evidence that simulation studies can provide adequate results to inform decisions about MST designs.