Sensitivity of Global IRT Fit Indices to the Magnitude and Prevalence of Model-Data Discrepancies.

Abstract

This study examined the operating characteristics of global fit indices for dichotomous item response theory (IRT) models, including the limited-information $M_2$ statistic, root mean square error of approximation (RMSEA), standardized root mean squared residual (SRMSR), comparative fit index (CFI), and Tucker-Lewis index (TLI). A simulation varied sample size, global and localized three-parameter logistic (3PL)-to-two-parameter logistic misspecification, the magnitude and prevalence of surface- and pair-specific underlying local dependence (ULD), null-independence responses, and reversed items under free- or positive-slope estimation. The design included 61 conditions with 1,000 replications per condition, supplemented by large-sample reference analyses. $M_2$ rejection rates were reasonably close to nominal levels under correct specification and increased with stronger and more prevalent local dependence, but sensitivity remained limited for localized 3PL misspecification and weak ULD. RMSEA remained small across most conditions, SRMSR showed substantial sample-size dependence, and CFI and TLI were more responsive to broad or severe discrepancies than to localized violations. Reversed items produced little global discrepancy when negative discriminations were allowed but substantial discrepancy under positive-slope constraints. Overall, global fit indices are best interpreted as complementary screening measures and should be supplemented with targeted diagnostics and parameter inspection.