A New Paradigm for Polling

Abstract

. Scientific fields operate within paradigms that define problems and solutions for a community of researchers. The dominant paradigm in polling centers on random sampling, which is unfortunate because random sampling is, for all practical purposes, dead. The pollsters who try to produce random samples fail because hardly anyone responds. And more and more pollsters do not even try. The field therefore has folded weighting-type adjustments into the paradigm, but this too is unfortunate because weighting works only if we assume away important threats to sampling validity, threats that loom particularly large in the growing non-probability polling sector. This paper argues that the polling field needs to move to a more general paradigm built around the Meng (2018) equation that characterizes survey error for any sampling approach, including non-random samples. Moving to this new paradigm has two important benefits. First, this new paradigm elevates new insights, including the fact that survey error increases with population size when individuals’ decisions to respond are correlated with how they respond. This insight helps us understand how small sampling defects can metastasize into large survey errors. Second, the new paradigm points the field toward new methods that more directly identify and account for sampling defects in a non-random sampling environment. This paper describes the intuition and potential power of these new tools, tools that are further elaborated in Bailey (2023b).