Comparing and Combining IRTree Models and Anchoring Vignettes in Addressing Response Styles

Abstract

Response styles pose great threats to psychological measurements. This research compares IRTree models and anchoring vignettes in addressing response styles and estimating the target traits. It also explores the potential of combining them at the item level and total-score level (ratios of extreme and middle responses to vignettes). Four models were evaluated: three multidimensional IRTree models with different levels of using vignette data and a nominal response model (NRM) addressing extreme and midpoint response styles with item-level vignette responses. Simulation results indicated that the IRTree model using item-level vignette responses outperformed others in estimating the target trait and response styles to different extents, with performance improving as the number of vignettes increased. Empirical findings further demonstrated that models using item-level vignette information yielded higher reliability and closely aligned target trait estimates. These results underscore the value of integrating anchoring vignettes with IRTree models to enhance estimation accuracy and control for response styles.