Too Anecdotal to Be True? Mechanical Turk Is Not All Bots and Bad Data: Response to Webb and Tangney (2022).

IF 10.5 1区心理学 Q1 PSYCHOLOGY, MULTIDISCIPLINARY

Perspectives on Psychological Science Pub Date : 2024-11-01 Epub Date: 2024-03-07 DOI:10.1177/17456916241234328

Melissa G Keith, Alexander S McKay

引用次数: 0

Abstract

In response to Webb and Tangney (2022) we call into question the conclusion that data collected on Amazon's Mechanical Turk (MTurk) was "at best-only 2.6% valid" (p. 1). We suggest that Webb and Tangney made certain choices during the study-design and data-collection process that adversely affected the quality of the data collected. As a result, the anecdotal experience of these authors provides weak evidence that MTurk provides low-quality data as implied. In our commentary we highlight best practice recommendations and make suggestions for more effectively collecting and screening online panel data.

查看原文本刊更多论文

轶事太假？Mechanical Turk并非都是机器人和坏数据：对 Webb 和 Tangney（2022 年）的回应》。

作为对 Webb 和 Tangney（2022 年）的回应，我们对亚马逊的 Mechanical Turk (MTurk) 上收集的数据 "最多只有 2.6% 有效"（第 1 页）这一结论提出质疑。我们认为，Webb 和 Tangney 在研究设计和数据收集过程中做出的某些选择对所收集数据的质量产生了不利影响。因此，这些作者的亲身经历无法证明 MTurk 所提供的数据质量低下。在我们的评论中，我们强调了最佳实践建议，并提出了更有效地收集和筛选在线小组数据的建议。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Perspectives on Psychological Science PSYCHOLOGY, MULTIDISCIPLINARY-

CiteScore

22.70

自引率

4.00%

发文量

111

期刊介绍： Perspectives on Psychological Science is a journal that publishes a diverse range of articles and reports in the field of psychology. The journal includes broad integrative reviews, overviews of research programs, meta-analyses, theoretical statements, book reviews, and articles on various topics such as the philosophy of science and opinion pieces about major issues in the field. It also features autobiographical reflections of senior members of the field, occasional humorous essays and sketches, and even has a section for invited and submitted articles. The impact of the journal can be seen through the reverberation of a 2009 article on correlative analyses commonly used in neuroimaging studies, which still influences the field. Additionally, a recent special issue of Perspectives, featuring prominent researchers discussing the "Next Big Questions in Psychology," is shaping the future trajectory of the discipline. Perspectives on Psychological Science provides metrics that showcase the performance of the journal. However, the Association for Psychological Science, of which the journal is a signatory of DORA, recommends against using journal-based metrics for assessing individual scientist contributions, such as for hiring, promotion, or funding decisions. Therefore, the metrics provided by Perspectives on Psychological Science should only be used by those interested in evaluating the journal itself.