A Comparison of Anchor Selection Strategies for DIF Analysis

Abstract

When no prior information of differential item functioning (DIF) exists for items in a test, either the rank-based or iterative purification procedure might be preferred. The rank-based purification selects anchor items based on a preliminary DIF test. For a preliminary DIF test, likelihood ratio test (LRT) based approaches (e.g., all-others-as-anchors: AOAA and one-item-anchor: OIA) and an improved version of Lord's Wald test (i.e., anchor-all-test-all: AATA) have been used in research studies. However, both LRT- and Wald-based procedures often select DIF items as anchor items and as a result, inflate Type $I$ error rates. To overcome this issue, minimum test statistics ($\mathrm{Min}{G}^2$/${\chi }^2$) or items with nonsignificant test statistics and large discrimination parameter estimates ($\mathrm{NonsigMax}$A) have been suggested in the literature to select anchor items. Nevertheless, little research has been done comparing combinations of the three anchor selection procedures paired with the two anchor selection criteria. Thus, the performance of the six rank-based strategies was compared in this study in terms of accuracy, power, and Type $I$ error rates. Among the rank-based strategies, the AOAA-based strategies demonstrated greater robustness across various conditions compared to the AATA- and OIA-based strategies. Additionally, the $\mathrm{Min}{G}^2$/${\chi }^2$ criterion exhibited better performance under various conditions compared to $\mathrm{NonsigMax}$A criterion.