Survey Mode and Item Non-Substantive Response: Insights from a Replication Study and Other Recent Trials

Abstract

This is a replication of a Survey Practice article that was published in September 2022 that explored survey administration mode and non-substantive response. It revisits the findings from the first study using a randomized controlled design. In this study, we found that on average phone respondents provided more non-substantive responses than web survey respondents, though the portion of phone and web respondents with one or more non-substantive response was similar. Another paper we published in Survey Practice in 2023 also explored mode effects with random assignment to text-to-web and phone groups. Thus, this is the third article by the authors on this topic released in Survey Practice. As part of this paper, we discuss findings from all three articles and contextualize them in relationship to other research on item non-substantive response and survey mode (specifically web and phone).