Variable-length fully Bayesian adaptive testing and associated stopping criteria.

Abstract

Many computerized adaptive testing (CAT) systems treat item parameters as if they were known without error, relying on point estimates obtained during item pool calibration. This practice can underestimate uncertainty in ability estimates and affect when a variable-length CAT terminates. A fully Bayesian (FB) CAT algorithm addresses this issue by explicitly incorporating item parameter uncertainty into both ability estimation and item selection. This study investigated the performance of FB CAT in a variable-length setting and compared it with conventional CAT under three stopping rules: a standard error (SE) rule, a change-in- $\theta$ (CIT) rule, and a combined CIT+SE rule. Simulation studies were conducted across a range of calibration sample sizes and item pool sizes. Results showed that the FB algorithm generally improved estimation accuracy and produced interval coverage rates closer to nominal levels, especially when the calibration sample size was small. The combined CIT+SE rule reduced unnecessarily long tests that can arise when using only the SE rule, particularly at $\theta$ levels for which the remaining item pool provides limited additional information to further reduce SE. Overall, the findings indicate that FB variable-length CAT can enhance uncertainty quantification, and that the combined CIT+SE rule offers a practical balance between measurement precision and testing efficiency.