Item fit statistics for Rasch analysis: can we trust them?

Abstract

To compare fit statistics for the Rasch model based on estimates of unconditional or conditional response probabilities. Using person estimates to calculate fit statistics can lead to problems because the person estimates are biased. Conditional response probabilities given the total person score could be used instead. Data sets are simulated which fit the Rasch model. Type I error rates are calculated and the distributions of the fit statistics are compared with the assumed normal or chi-square distribution. Parametric bootstrap is used to further study the distributions of the fit statistics. Type I error rates for unconditional chi-square statistics are larger than expected even for moderate sample sizes. The conditional chi-square statistics maintain the significance level. Unconditional outfit and infit statistics have asymmetric distributions with means slighly below 1. Conditional outfit and infit statistics have reduced Type I error rates. Conditional residuals should be used. If only unconditional residuals are available parametric bootstrapping is recommended to calculate valid p-values. Bootstrapping is also necessary for conditional outfit statistics. For conditional infit statistics the adjusted rule-of-thumb critical values look useful.