Evaluating nonresponse and non-sampling error trends in election studies

Abstract

Political scientists rely on election studies as high-quality sources of data on voting behavior and attitudes. However, despite a cross-national trend of declining response rates over time and a lively debate regarding the accuracy of pre-election polls, data quality in election studies is an underdiscussed topic. In this paper, we address this research gap by focusing on trends in survey participation and non-sampling errors over time using data from the Icelandic National Election Study over a period spanning nearly four decades (1983–2021). We find that response rates have halved in the period under study (from around 70 % to 36 %), caused by increasing noncontact rates. Focusing on sample composition, we find that response rates have declined more among young adults and those without university degrees than other sample subgroups. To assess non-sampling error trends, we propose a simple metric based on the mean average error (MAE), which accounts for the number of parties and the sample size. Surprisingly, we find that despite decreasing response rates, the MAE has not increased, and for most elections, we cannot rule out sampling error alone as the explanation for the MAE. Finally, we show that adjustment weights have small and inconsistent effects on the MAE, suggesting that the auxiliary information available in the Icelandic context lacks the strong correlations needed to reduce error in the estimation of vote choice. We conclude with a discussion of these findings, their implications, and some guidance for practitioners seeking to evaluate data quality that can inform changes to the design of election studies.