一项准实验研究,探讨多模式僵尸筛查工具的功效,以及在在线心理研究中维护数据完整性的建议。

IF 12.3 1区 心理学 Q1 PSYCHOLOGY, MULTIDISCIPLINARY
American Psychologist Pub Date : 2024-10-01 Epub Date: 2023-07-20 DOI:10.1037/amp0001183
Melissa Simone, Cory J Cascalheira, Benjamin G Pierce
{"title":"一项准实验研究,探讨多模式僵尸筛查工具的功效,以及在在线心理研究中维护数据完整性的建议。","authors":"Melissa Simone, Cory J Cascalheira, Benjamin G Pierce","doi":"10.1037/amp0001183","DOIUrl":null,"url":null,"abstract":"<p><p>Bots are automated software programs that pose an ongoing threat to psychological research by invading online research studies and their increasing sophistication over time. Despite this growing concern, research in this area has been limited to bot detection in existing data sets following an unexpected encounter with bots. The present three-condition, quasi-experimental study aimed to address this gap in the literature by examining the efficacy of three types of bot screening tools across three incentive conditions ($0, $1, and $5). Data were collected from 444 respondents via Twitter advertisements between July and September 2021. The efficacy of five <i>task-based</i> (i.e., anagrams, visual search), <i>question-based</i> (i.e., attention checks, ReCAPTCHA), and <i>data-based</i> (i.e., consistency, metadata) tools was examined with Bonferroni-adjusted univariate and multivariate logistic regression analyses. In general, study results suggest that bot screening tools function similarly for participants recruited across incentive conditions. Moreover, the present analyses revealed heterogeneity in the efficacy of bot screening tool subtypes. Notably, the present results suggest that the least effective bot screening tools were among the most commonly used tools in existing literature (e.g., ReCAPTCHA). In sum, the study findings revealed highly effective and highly ineffective bot screening tools. Study design and data integrity recommendations for researchers are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>","PeriodicalId":48468,"journal":{"name":"American Psychologist","volume":" ","pages":"956-969"},"PeriodicalIF":12.3000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10799166/pdf/","citationCount":"0","resultStr":"{\"title\":\"A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research.\",\"authors\":\"Melissa Simone, Cory J Cascalheira, Benjamin G Pierce\",\"doi\":\"10.1037/amp0001183\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Bots are automated software programs that pose an ongoing threat to psychological research by invading online research studies and their increasing sophistication over time. Despite this growing concern, research in this area has been limited to bot detection in existing data sets following an unexpected encounter with bots. The present three-condition, quasi-experimental study aimed to address this gap in the literature by examining the efficacy of three types of bot screening tools across three incentive conditions ($0, $1, and $5). Data were collected from 444 respondents via Twitter advertisements between July and September 2021. The efficacy of five <i>task-based</i> (i.e., anagrams, visual search), <i>question-based</i> (i.e., attention checks, ReCAPTCHA), and <i>data-based</i> (i.e., consistency, metadata) tools was examined with Bonferroni-adjusted univariate and multivariate logistic regression analyses. In general, study results suggest that bot screening tools function similarly for participants recruited across incentive conditions. Moreover, the present analyses revealed heterogeneity in the efficacy of bot screening tool subtypes. Notably, the present results suggest that the least effective bot screening tools were among the most commonly used tools in existing literature (e.g., ReCAPTCHA). In sum, the study findings revealed highly effective and highly ineffective bot screening tools. Study design and data integrity recommendations for researchers are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>\",\"PeriodicalId\":48468,\"journal\":{\"name\":\"American Psychologist\",\"volume\":\" \",\"pages\":\"956-969\"},\"PeriodicalIF\":12.3000,\"publicationDate\":\"2024-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10799166/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"American Psychologist\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.1037/amp0001183\",\"RegionNum\":1,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2023/7/20 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"PSYCHOLOGY, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Psychologist","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/amp0001183","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/7/20 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

摘要

机器人是一种自动化软件程序,它入侵在线研究,并随着时间的推移变得越来越复杂,从而对心理研究构成了持续的威胁。尽管这一问题日益受到关注,但该领域的研究仅限于在意外遭遇机器人后在现有数据集中进行机器人检测。本项三条件准实验研究旨在通过考察三种类型的僵尸筛选工具在三种激励条件(0 美元、1 美元和 5 美元)下的功效,填补文献中的这一空白。研究人员在 2021 年 7 月至 9 月期间通过 Twitter 广告从 444 名受访者处收集了数据。通过 Bonferroni-adjusted 单变量和多变量逻辑回归分析,考察了五种基于任务(即猜谜语、视觉搜索)、基于问题(即注意力检查、ReCAPTCHA)和基于数据(即一致性、元数据)的工具的有效性。总体而言,研究结果表明,机器人筛查工具对不同激励条件下招募的参与者具有相似的功能。此外,本分析还揭示了机器人筛查工具亚型功效的异质性。值得注意的是,本研究结果表明,效果最差的机器人筛选工具属于现有文献中最常用的工具(如 ReCAPTCHA)。总之,研究结果揭示了高度有效和高度无效的僵尸筛选工具。本研究还为研究人员提供了研究设计和数据完整性方面的建议。(PsycInfo 数据库记录 (c) 2023 APA,保留所有权利)。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research.

Bots are automated software programs that pose an ongoing threat to psychological research by invading online research studies and their increasing sophistication over time. Despite this growing concern, research in this area has been limited to bot detection in existing data sets following an unexpected encounter with bots. The present three-condition, quasi-experimental study aimed to address this gap in the literature by examining the efficacy of three types of bot screening tools across three incentive conditions ($0, $1, and $5). Data were collected from 444 respondents via Twitter advertisements between July and September 2021. The efficacy of five task-based (i.e., anagrams, visual search), question-based (i.e., attention checks, ReCAPTCHA), and data-based (i.e., consistency, metadata) tools was examined with Bonferroni-adjusted univariate and multivariate logistic regression analyses. In general, study results suggest that bot screening tools function similarly for participants recruited across incentive conditions. Moreover, the present analyses revealed heterogeneity in the efficacy of bot screening tool subtypes. Notably, the present results suggest that the least effective bot screening tools were among the most commonly used tools in existing literature (e.g., ReCAPTCHA). In sum, the study findings revealed highly effective and highly ineffective bot screening tools. Study design and data integrity recommendations for researchers are provided. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
American Psychologist
American Psychologist PSYCHOLOGY, MULTIDISCIPLINARY-
CiteScore
18.50
自引率
1.20%
发文量
145
期刊介绍: Established in 1946, American Psychologist® is the flagship peer-reviewed scholarly journal of the American Psychological Association. It publishes high-impact papers of broad interest, including empirical reports, meta-analyses, and scholarly reviews, covering psychological science, practice, education, and policy. Articles often address issues of national and international significance within the field of psychology and its relationship to society. Published in an accessible style, contributions in American Psychologist are designed to be understood by both psychologists and the general public.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信