Should we include margins of error in public opinion polls?

Abstract

Public opinion polls have become vital and increasingly visible parts of election campaigns. Previous research has frequently demonstrated that polls can influence both citizens' voting intentions and political parties' campaign strategies. However, they are also fraught with uncertainty. Margins of error can reflect (parts of) this uncertainty. This paper investigates how citizens' voting intentions change due to whether polling estimates are presented with or without margins of error.

Using a vignette experiment (N=3224), we examine this question based on a real-world example in which different election polls were shown to nationally representative respondents ahead of the 2021 federal election in Germany. We manipulated the display of the margins of error, the interpretation of polls and the closeness of the electoral race.

The results indicate that margins of error can influence citizens' voting intentions. This effect is dependent on the actual closeness of the race and additional interpretative guidance provided to voters. More concretely, the results consistently show that margins of error increase citizens' inclination to vote for one of the two largest contesting parties if the polling gap between these parties is small, and an interpretation underlines this closeness.

The findings of this study are important for three reasons. First, they help to determine whether margins of error can assist citizens in making more informed (strategic) vote decisions. They shed light on whether depicting opinion-poll uncertainty affects the key features of representative democracy, such as democratic accountability. Second, the results stress the responsibility of the media. The way polls are interpreted and contextualized influences the effect of margins of error on voting behaviour. Third, the findings of this paper underscore the significance of including methodological details when communicating scientific research findings to the broader public.