The Precision of Ability Estimates and Item Pool Utilization Depending on Degrees of Correlations among Subsections and Item Selection Criteria in Computerized Adaptive Testing

Abstract

The purpose of this study was to investigate the effects of degrees of correlations among subsections and item selection criteria on the precision of ability estimates and item pool utilization in computerized adaptive testing (CAT). Based on the results of this simulation study, the following conclusions can be made. First, the precision of ability estimates would be decreased when the correlations among subsections was relatively low. Second, when implementing an a-stratified item selection method (AST), the effects of degrees of correlations among subsections on the precision of ability estimates increased, compared to the maximum fisher information item selection method (MFI). Third, when the test length was relatively long, the effects of degrees of correlations among subsections on the precision of ability estimates decreased. Forth, MFI with content-balancing increased the precision of ability estimates, especially in the case of low correlations among subsections. The item pool utilization seemed not to be considerably affected by the correlations among subsections, implementing a-stratification item selection method and/or item exposure strategy, the item pool utilization increased.