Using Cognitive Interview Data to Assess Data Quality and the Cultural Norms of Survey Participants

Cognitive interviews have become the gold standard in the federal statistical system for evaluating the performance of surveys. The current study draws on cognitive interview data to demonstrate how cognitive interviews can be used to gain a deeper understanding of the cultural norms of survey participants. Drawing on interviews conducted in preparation for the 2017 Census of Agriculture, this study identifies the disproportionate rates by which male and female agricultural operators are underreported on the form, and the cultural underpinnings of these patterns of reporting. While male agricultural operators are more likely to be underreported because of differentials in decision-making authority within the operation, females are more likely to be underreported for arbitrary reasons. The value of understanding the cultural norms in which participants are embedded and their effects on the quality of the data provided by respondents is discussed.