Measuring Agreement: How to Arrive at Reliable Measures of Opinion Congruence Between Voters and Parties

ABSTRACT The extent to which voters and parties agree on policies is an important way through which political scientists have empirically studied political representation. This opinion congruence is most often measured by comparing preferences on a number of policy statements. While the selection of policy statements has not escaped scholarly attention, its impact on the reliability of congruence scores, i.e. the degree to which similar levels of opinion congruence are found when different samples of policy statements are used, has been less investigated. This article looks at which factors of statements samples and voters affect the reliability of congruence measures. It does so by simulating over 5 million opinion congruence scores on the basis of a dataset containing 134 voter and party policy preferences. It finds that both the number of statements and their topic diversity positively affect the reliability of congruence estimates. In addition, the congruence estimates of politically less sophisticated voters are more reliable but only when many left-right policy statements are included in the statement selection. Finally, explorative analyses suggest that increasing topic diversity also increases the validity of congruence measures.