Improving Statistical Analysis in Team Science: The Case of a Bayesian Multiverse of Many Labs 4

IF 15.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science Pub Date : 2023-07-01 DOI:10.1177/25152459231182318

S. Hoogeveen, S. Berkhout, Q. Gronau, E. Wagenmakers, J. Haaf

{"title":"Improving Statistical Analysis in Team Science: The Case of a Bayesian Multiverse of Many Labs 4","authors":"S. Hoogeveen, S. Berkhout, Q. Gronau, E. Wagenmakers, J. Haaf","doi":"10.1177/25152459231182318","DOIUrl":null,"url":null,"abstract":"Team-science projects have become the “gold standard” for assessing the replicability and variability of key findings in psychological science. However, we believe the typical meta-analytic approach in these projects fails to match the wealth of collected data. Instead, we advocate the use of Bayesian hierarchical modeling for team-science projects, potentially extended in a multiverse analysis. We illustrate this full-scale analysis by applying it to the recently published Many Labs 4 project. This project aimed to replicate the mortality-salience effect—that being reminded of one’s own death strengthens the own cultural identity. In a multiverse analysis, we assess the robustness of the results with varying data-inclusion criteria and prior settings. Bayesian model comparison results largely converge to a common conclusion: The data provide evidence against a mortality-salience effect across the majority of our analyses. We issue general recommendations to facilitate full-scale analyses in team-science projects.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":15.6000,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advances in Methods and Practices in Psychological Science","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/25152459231182318","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY","Score":null,"Total":0}

引用次数: 1

Abstract

Team-science projects have become the “gold standard” for assessing the replicability and variability of key findings in psychological science. However, we believe the typical meta-analytic approach in these projects fails to match the wealth of collected data. Instead, we advocate the use of Bayesian hierarchical modeling for team-science projects, potentially extended in a multiverse analysis. We illustrate this full-scale analysis by applying it to the recently published Many Labs 4 project. This project aimed to replicate the mortality-salience effect—that being reminded of one’s own death strengthens the own cultural identity. In a multiverse analysis, we assess the robustness of the results with varying data-inclusion criteria and prior settings. Bayesian model comparison results largely converge to a common conclusion: The data provide evidence against a mortality-salience effect across the majority of our analyses. We issue general recommendations to facilitate full-scale analyses in team-science projects.

查看原文本刊更多论文

改进团队科学中的统计分析:多实验室贝叶斯多元宇宙的案例

团队科学项目已成为评估心理科学关键发现的可复制性和可变性的“金标准”。然而，我们认为这些项目中典型的元分析方法无法与收集到的丰富数据相匹配。相反，我们提倡在团队科学项目中使用贝叶斯层次建模，这可能会扩展到多元宇宙分析中。我们通过将其应用于最近发布的Many Labs 4项目来说明这种全面的分析。这个项目旨在复制死亡显著性效应——提醒自己的死亡会加强自己的文化认同。在多元宇宙分析中，我们评估了不同数据包含标准和先前设置的结果的稳健性。贝叶斯模型的比较结果在很大程度上得出了一个共同的结论：这些数据为我们大多数分析中的死亡率显著性效应提供了证据。我们发布一般性建议，以促进团队科学项目的全面分析。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Advances in Methods and Practices in Psychological Science Multiple-

CiteScore

21.20

自引率

0.70%

发文量

期刊介绍： In 2021, Advances in Methods and Practices in Psychological Science will undergo a transition to become an open access journal. This journal focuses on publishing innovative developments in research methods, practices, and conduct within the field of psychological science. It embraces a wide range of areas and topics and encourages the integration of methodological and analytical questions. The aim of AMPPS is to bring the latest methodological advances to researchers from various disciplines, even those who are not methodological experts. Therefore, the journal seeks submissions that are accessible to readers with different research interests and that represent the diverse research trends within the field of psychological science. The types of content that AMPPS welcomes include articles that communicate advancements in methods, practices, and metascience, as well as empirical scientific best practices. Additionally, tutorials, commentaries, and simulation studies on new techniques and research tools are encouraged. The journal also aims to publish papers that bring advances from specialized subfields to a broader audience. Lastly, AMPPS accepts Registered Replication Reports, which focus on replicating important findings from previously published studies. Overall, the transition of Advances in Methods and Practices in Psychological Science to an open access journal aims to increase accessibility and promote the dissemination of new developments in research methods and practices within the field of psychological science.