The Effect of Response Formats on Response Style Strength

Abstract

Abstract: Many researchers use self-report data to examine abilities, personalities, or attitudes. At the same time, there is a widespread concern that response styles, such as the tendency to give extreme, midscale, or acquiescent responses, may threaten data quality. As an alternative to post hoc control of response styles using psychometric models, a priori control using specific response formats may be a means to reduce biasing response style effects in self-report data in day-to-day research practice. Previous research has suggested that response styles were less influential in a Drag-and-Drop (DnD) format compared to the traditional Likert-type format. In this article, we further examine the advantage of the DnD format, test its generalizability, and investigate its underlying mechanisms. In two between-participants experiments, we tested different versions of the DnD format against the Likert format. We found no evidence for reduced response style influence in any of the DnD conditions, nor did we find any difference between the conditions in terms of the validity of the measures to external criteria. We conclude that adaptations of response formats, such as the DnD format, may be promising, but require more thorough examination before recommending them as a means to reduce response style influence in psychological measurement.