Methodological considerations for the design and implementation of reliable and valid web surveys

Abstract

This article examines practical epidemiology principles related to the design and implementation of web surveys. Six practice-oriented items are critically examined: 1) The study question; 2) The target population; 3) Study population needed; 4) Sampling or selecting the participants in a representative manner; 5) Sending the survey invitations in a manner that is efficient, safe, and mitigates bias; and 6) Assessing and enhancing the external validity of collected data. Clearly articulating the study’s purpose (including whether there is an intent to create generalizable knowledge) influences the sampling approach: probabilistic or non-probabilistic. Similarly, properly defining the study population (people, place and time) prevents overgeneralization of study findings. Adjustments to sample size may be needed to address different real-world complexities, including multi-purpose surveys with different (possibly un-related outcomes), multiple target populations, subgroup analyses, and cluster sampling. When the sample is being drawn from a sampling frame, efforts must be made to ensure that the frame is complete, current, and correct to reduce under-sampling. The choice of environment in which data collection is hosted is critical; practical considerations include data volume, variety, vulnerability, and the software’s capabilities and cost. Although web surveys, in general, are becoming increasingly easier to conduct, good web surveys in contrast are becoming increasingly harder to undertake. Careful consideration should be given to sampling and nonsampling sources of error when designing web surveys to ensure validity and reliability.