Posterior predictive checks for the detection of extreme response style.

Abstract

Extreme response style (ERS), the tendency of participants to select extreme item categories regardless of the item content, has frequently been found to decrease the validity of Likert-type questionnaire results (e.g., Moors, European Journal of Work and Organizational Psychology, 21, 271-298, 2012). For this reason, detecting ERS at both the group and individual levels is of paramount importance. While various approaches to detecting ERS exist, these may conflate ERS with the trait of interest, require additional questionnaires to be administered, or require the use of mixture or multidimensional IRT models. As an alternative approach to detecting ERS, Bayesian posterior predictive checks (PPCs) may be a viable option. Posterior predictive checking offers a highly customizable framework for detecting model misfit, which can be directly applied to frequently used unidimensional IRT models. Critically, the use of PPCs to detect ERS does not require strong assumptions regarding the nature of ERS, such as ERS being a continuous dimension or a categorical trait. In this paper, we thus apply PPCs to a generalized partial credit model to detect model misfit related to ERS on both the group and person levels. We propose various possible PPCs tailored to ERS, which are illustrated in an empirical example, and their performance in detecting ERS is examined under various conditions. Suggestions for practical applications are provided, and avenues for future research are explored.