How to mitigate selection bias in COVID-19 surveys: evidence from five national cohorts

Abstract

Non-response to surveys is a common problem; even more so during the COVID-19 pandemic with social distancing measures challenging data collection. As respondents often differ from non-respondents, this can introduce bias. The goal of the current study was to see if we can reduce bias and restore sample representativeness in a series of COVID-19 surveys embedded within five UK cohort studies by using the rich data available from previous waves of data collection. Three surveys were conducted during the pandemic across five UK cohorts: National Survey of Health and Development (NSHD, born 1946), 1958 National Child Development Study (NCDS), 1970 British Cohort Study (BCS70), Next Steps (born 1989-90) and Millennium Cohort Study (MCS, born 2000-02). Response rates in the COVID-19 surveys were lower compared to previous waves, especially in the younger cohorts. We identified bias due to systematic non-response in several variables, with more respondents in the most advantaged social class and among those with higher childhood cognitive ability. Making use of the rich data available pre-pandemic in these longitudinal studies, the application of non-response weights and multiple imputation was successful in reducing bias in parental social class and childhood cognitive ability, nearly eliminating it for the former. Surveys embedded within existing cohort studies offer a clear advantage over cross-sectional samples collected during the pandemic in terms of their ability to mitigate selection bias. This will enhance the quality and reliability of future research studying the medium and long-term effects of the pandemic.