Assessing the external validity of mobile phone surveys for monitoring ITN coverage indicators: a comparison with household surveys in Tanzania.

Abstract

Background: Insecticide-treated net (ITN) coverage indicators allow country malaria programmes to understand the overall coverage of their ITN distribution activities and can be used to forecast the need for additional ITN procurement and to plan future distribution campaigns. As a result, more frequent data collection could better guide programme strategies, particularly for those strategies operating on an annual framework; however, the high costs and infrequency of national-scale, household surveys limit their practicality for ongoing monitoring. Due to the rapid growth trend of mobile phone ownership in low- and middle-income countries in recent years, mobile phone-based surveys (MPS) have emerged as a comparatively inexpensive alternative to large-scale household surveys.

Methods: Contemporaneous to the 2017 Malaria Indicator Survey and the 2022 Demographic and Health Survey, a random digit dial and list-based MPS were conducted in Tanzania, respectively. All surveys allowed for the calculation of the same ITN coverage indicators at the regional level which permitted an assessment of the external validity of the MPS.

Results: Mean measures of absolute bias, as derived from Bland-Altman analyses, for survey raked estimates of households with at least one bed net of any type and household population with access to a bed net of any type were negative for both surveys. These MPS estimates were, on average, consistently lower than corresponding estimates from the household surveys by 7-8 percentage points. Proportion of households with at least one bed net of any type for every two people was unbiased with measures of 0.02 [95% CI - 0.04, 0.08] for the initial MPS and 0.02 [95% CI - 0.03, 0.06] for the 2022 MPS, following survey raking, relative to the household surveys.

Conclusions: MPS consistently underestimated household ITN access compared to contemporaneous household surveys, but the direction and magnitude of bias were stable across years. While this limits their use as a direct substitute for household surveys, MPS may still offer timely, conservative signals of declining coverage where other data are delayed or unavailable. With further validation and potential adjustment for bias, MPS could complement traditional monitoring approaches, especially in contexts where rapid data are needed to guide programme decisions.