Measuring electoral democracy with observables

Abstract

Most cross-national indices of democracy rely centrally on coder judgments, which are susceptible to bias and error, and require expensive and time-consuming coding by experts. We present an approach to measurement based on observables that aim to preserve the nuanced quality of subjectively coded democracy indices. Our observable-to-subjective score mapping is free of idiosyncratic coder errors arising from misinformation, slack, or biases. It is less susceptible to systematic bias that may arise from coders’ inferences about a country's regime, for example, from the ideology of the ruler. The data collection procedure and mode of analysis are fully transparent and replicable, and the procedure is based on random forests and is cheap to produce, easy to update, and offers coverage for all polities with sovereign or semisovereign status, surpassing the sample of any existing index. We show that this expansive coverage makes a big difference to our understanding of some causal questions.