Does survey mode matter? An experimental evaluation of data quality in China

Abstract

This study employs experimental data from the Labor Force Survey (LFS) to compare the data quality of computer-assisted web interviewing (CAWI) and computer-assisted personal interviewing (CAPI). The experiment was conducted in 2023 as part of the LFS, an ongoing longitudinal face-to-face survey of Chinese adults aged 16 and above in a mega city. Using the identical questionnaire, respondents were randomly assigned to either the control group (CAPI mode only) or the treatment group (optional CAPI and CAWI modes). The characteristics of households and individuals obtained using mixed-mode design do not significantly differ from those obtained from single-mode design (CAPI), indicating no mode effect on data quality for mixed-mode surveys. Additionally, there are no significant differences in data quality between CAWI and CAPI. CAWI respondents tend to take more time to answer the questions compared to CAPI respondents. Our findings offer valuable insights into enhancing mode-specific targeting and improving the quality of survey data collection by leveraging both existing survey data and paradata.