在线调查中的欺诈:来自非概率亚总体样本的证据

IF 3.2 Q1 POLITICAL SCIENCE

Journal of Experimental Political Science Pub Date : 2022-05-27 DOI:10.1017/XPS.2022.8

Andrew M. Bell, Thomas Gift

{"title":"在线调查中的欺诈:来自非概率亚总体样本的证据","authors":"Andrew M. Bell, Thomas Gift","doi":"10.1017/XPS.2022.8","DOIUrl":null,"url":null,"abstract":"Abstract We hired a well-known market research firm whose surveys have been published in leading political science journals, including JEPS. Based on a set of rigorous “screeners,” we detected what appears to be exceedingly high rates of identity falsification: over 81 percent of respondents seemed to misrepresent their credentials to gain access to the survey and earn compensation. Similarly high rates of presumptive character falsification were present in panels from multiple sub-vendors procured by the firm. Moreover, we found additional, serious irregularities embedded in the data, including evidence of respondents using deliberate strategies to detect and circumvent one of our screeners, as well as pervasive, observable patterns reflecting that the survey had been taken repeatedly by a respondent or collection of respondents. This evidence offers reasons to be concerned about the quality of online nonprobability, subpopulation samples, and calls for further, systematic research.","PeriodicalId":37558,"journal":{"name":"Journal of Experimental Political Science","volume":null,"pages":null},"PeriodicalIF":3.2000,"publicationDate":"2022-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Fraud in Online Surveys: Evidence from a Nonprobability, Subpopulation Sample\",\"authors\":\"Andrew M. Bell, Thomas Gift\",\"doi\":\"10.1017/XPS.2022.8\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract We hired a well-known market research firm whose surveys have been published in leading political science journals, including JEPS. Based on a set of rigorous “screeners,” we detected what appears to be exceedingly high rates of identity falsification: over 81 percent of respondents seemed to misrepresent their credentials to gain access to the survey and earn compensation. Similarly high rates of presumptive character falsification were present in panels from multiple sub-vendors procured by the firm. Moreover, we found additional, serious irregularities embedded in the data, including evidence of respondents using deliberate strategies to detect and circumvent one of our screeners, as well as pervasive, observable patterns reflecting that the survey had been taken repeatedly by a respondent or collection of respondents. This evidence offers reasons to be concerned about the quality of online nonprobability, subpopulation samples, and calls for further, systematic research.\",\"PeriodicalId\":37558,\"journal\":{\"name\":\"Journal of Experimental Political Science\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2022-05-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Experimental Political Science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1017/XPS.2022.8\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"POLITICAL SCIENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Experimental Political Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1017/XPS.2022.8","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"POLITICAL SCIENCE","Score":null,"Total":0}

引用次数: 0

摘要

摘要我们聘请了一家知名的市场研究公司，该公司的调查结果已发表在包括JEPS在内的主流政治科学期刊上。基于一组严格的“筛选者”，我们发现了似乎极高的身份伪造率：超过81%的受访者似乎歪曲了自己的证件，以获得调查并获得赔偿。该公司采购的多个次级供应商的面板中也存在同样高的假定性格伪造率。此外，我们发现数据中存在其他严重的违规行为，包括受访者使用故意策略来检测和规避我们的一名筛选人员的证据，以及普遍存在的、可观察到的模式，反映出调查是由一名受访者或一组受访者反复进行的。这一证据提供了关注在线不可能性、亚群体样本质量的理由，并呼吁进行进一步的系统研究。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Fraud in Online Surveys: Evidence from a Nonprobability, Subpopulation Sample

Abstract We hired a well-known market research firm whose surveys have been published in leading political science journals, including JEPS. Based on a set of rigorous “screeners,” we detected what appears to be exceedingly high rates of identity falsification: over 81 percent of respondents seemed to misrepresent their credentials to gain access to the survey and earn compensation. Similarly high rates of presumptive character falsification were present in panels from multiple sub-vendors procured by the firm. Moreover, we found additional, serious irregularities embedded in the data, including evidence of respondents using deliberate strategies to detect and circumvent one of our screeners, as well as pervasive, observable patterns reflecting that the survey had been taken repeatedly by a respondent or collection of respondents. This evidence offers reasons to be concerned about the quality of online nonprobability, subpopulation samples, and calls for further, systematic research.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Journal of Experimental Political Science Social Sciences-Sociology and Political Science

CiteScore

5.10

自引率

8.30%

发文量

期刊介绍： The Journal of Experimental Political Science (JEPS) features cutting-edge research that utilizes experimental methods or experimental reasoning based on naturally occurring data. We define experimental methods broadly: research featuring random (or quasi-random) assignment of subjects to different treatments in an effort to isolate causal relationships in the sphere of politics. JEPS embraces all of the different types of experiments carried out as part of political science research, including survey experiments, laboratory experiments, field experiments, lab experiments in the field, natural and neurological experiments. We invite authors to submit concise articles (around 4000 words or fewer) that immediately address the subject of the research. We do not require lengthy explanations regarding and justifications of the experimental method. Nor do we expect extensive literature reviews of pros and cons of the methodological approaches involved in the experiment unless the goal of the article is to explore these methodological issues. We expect readers to be familiar with experimental methods and therefore to not need pages of literature reviews to be convinced that experimental methods are a legitimate methodological approach. We will consider longer articles in rare, but appropriate cases, as in the following examples: when a new experimental method or approach is being introduced and discussed or when novel theoretical results are being evaluated through experimentation. Finally, we strongly encourage authors to submit manuscripts that showcase informative null findings or inconsistent results from well-designed, executed, and analyzed experiments.