Effects of Push-To-Web Mixed Mode Approaches on Survey Response Rates: Evidence from a Randomized Experiment in Emergency Departments

Abstract

Recent work involving survey administration among emergency department (ED) patients has demonstrated very low response rates using single-mode (e.g., mail-only or web-only) approaches. In this study, the authors tested several mixed-mode protocols that focus on a push-to-web approach for survey administration in this challenging but important setting. A random sample of 26,991 ED discharged-to-community (DTC) patients discharged in January 2018–March 2018 from 16 hospital-based EDs nationwide were randomized to nine survey arms, eight of which involved a web survey with one or more of the following methods of invitation to the web survey: an emailed link to the web survey, a texted link to the web survey, and/or a mailed paper invitation with information on how to access the web survey. The reference arm was standard mixed mode (mailed survey with telephone follow-up). Invitation(s) to the web survey was followed by (1) a mailed survey, (2) telephone follow-up, or (3) both. The overall response rate across all arms was 18.6% (American Association for Public Opinion Research [AAPOR] RR1). None of the tested web-survey arms had significantly higher response rates than standard mixed mode (25.5%). Protocol comparisons demonstrated that both text outreach and including a telephone mode increased response rates. When examining response by completion mode, the arms involving text invitation had a higher percentage of completed surveys that were completed by web than arms not involving text invitation. Characteristics (e.g., age) of telephone and web respondents were more representative of the sampled population than mail respondents. Older patients and male patients were significantly less likely to be reachable by email or text. These results demonstrate that sequential multi-mode approaches involving web, mail, and telephone with multiple methods of web survey invitation show the greatest promise for obtaining higher response rates and more representative respondents from the ED population.