How Group Differences in Matching Criterion Distribution and IRT Item Difficulty Can Influence the Magnitude of the Mantel–Haenszel Chi-Square DIF Index

Abstract

Abstract The authors demonstrated the empirical relationship between Cohen's chi-square effect size, w, and differential item functioning (DIF), defined as group differences in item response theory (IRT) item difficulty. In Experiment 1, in which the lower asymptote was 0, the authors argued that Cohen's designation of small, medium, and large effects connotes reasonably well for that definition of DIF. In Experiment 2, the lower asymptote of the item response function was raised from 0 to 0.2 and the item discrimination parameter was held to 1.0. Doing so admitted non-crossing nonuniform DIF to the model, violating an underlying assumption of the Mantel–Haenszel procedure that the odds ratio is constant across studied levels of the matching criterion. Smaller difficulty parameter difference resulted, which produced larger effects with an inflation in effect size of about 15%. In Experiment 3, the authors used the 1-parameter logistic model to examine the effect that group differences in the matching crit...