Question-Order Effect in the Study of Satisfaction with Democracy: Lessons from Three Split-Ballot Experiments

IF 2.4 2区社会学 Q1 SOCIOLOGY

Sociological Methodology Pub Date : 2024-05-28 DOI:10.1177/00811750241254363

Zsófia Papp, Pál Susánszky, Andrea Szabó

{"title":"Question-Order Effect in the Study of Satisfaction with Democracy: Lessons from Three Split-Ballot Experiments","authors":"Zsófia Papp, Pál Susánszky, Andrea Szabó","doi":"10.1177/00811750241254363","DOIUrl":null,"url":null,"abstract":"This study examines question-order effects in measuring satisfaction with democracy (SWD). Particularly, the authors are interested in whether the relative position of the question regarding satisfaction with the state of the economy (SWE) in the questionnaire affects responses to the SWD item. The authors conducted three independent split-ballot experiments in Hungary between March 2021 and May 2022. They report a significant and substantial negative priming effect that possibly leads to a systematic underestimation of SWD. Importantly, the authors find no question-order effect in the measurement of SWE. The analysis further reveals a contrast effect: when the SWD question is primed, the difference between SWE and SWD means increases. The authors’ final recommendation is that researchers either put the SWD question before the SWE item to avoid question-order bias or randomize question order. These findings should assist future data collection efforts (comparative or single-country studies) in developing and integrating a battery of satisfaction items into questionnaires and help users assess data quality.","PeriodicalId":48140,"journal":{"name":"Sociological Methodology","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2024-05-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Sociological Methodology","FirstCategoryId":"90","ListUrlMain":"https://doi.org/10.1177/00811750241254363","RegionNum":2,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SOCIOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

This study examines question-order effects in measuring satisfaction with democracy (SWD). Particularly, the authors are interested in whether the relative position of the question regarding satisfaction with the state of the economy (SWE) in the questionnaire affects responses to the SWD item. The authors conducted three independent split-ballot experiments in Hungary between March 2021 and May 2022. They report a significant and substantial negative priming effect that possibly leads to a systematic underestimation of SWD. Importantly, the authors find no question-order effect in the measurement of SWE. The analysis further reveals a contrast effect: when the SWD question is primed, the difference between SWE and SWD means increases. The authors’ final recommendation is that researchers either put the SWD question before the SWE item to avoid question-order bias or randomize question order. These findings should assist future data collection efforts (comparative or single-country studies) in developing and integrating a battery of satisfaction items into questionnaires and help users assess data quality.

查看原文本刊更多论文

民主满意度研究中的问题顺序效应：三项分票实验的启示

本研究探讨了民主满意度（SWD）测量中的问题顺序效应。作者特别关注的是，有关经济状况满意度（SWE）的问题在问卷中的相对位置是否会影响对 SWD 项目的回答。作者于 2021 年 3 月至 2022 年 5 月期间在匈牙利进行了三次独立的分票实验。他们报告称，存在明显且大量的负引物效应，可能会导致对全部门发展的系统性低估。重要的是，作者发现在测量 SWE 时没有问题顺序效应。分析还进一步揭示了一种对比效应：当 SWD 问题被引出时，SWE 和 SWD 平均值之间的差异会增大。作者的最后建议是，研究人员要么将社会工作能力问题放在社会工作教育项目之前，以避免问题顺序偏差，要么随机调整问题顺序。这些发现应有助于未来的数据收集工作（比较研究或单一国家研究），帮助开发满意度项目并将其整合到问卷中，同时帮助用户评估数据质量。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Sociological Methodology SOCIOLOGY-

CiteScore

4.50

自引率

0.00%

发文量

期刊介绍： Sociological Methodology is a compendium of new and sometimes controversial advances in social science methodology. Contributions come from diverse areas and have something useful -- and often surprising -- to say about a wide range of topics ranging from legal and ethical issues surrounding data collection to the methodology of theory construction. In short, Sociological Methodology holds something of value -- and an interesting mix of lively controversy, too -- for nearly everyone who participates in the enterprise of sociological research.