Do not put all of your eggs in one basket: Multiverse analysis in applied psychology.

IF 9.4 1区心理学 Q1 MANAGEMENT

Journal of Applied Psychology Pub Date : 2025-05-05 DOI:10.1037/apl0001291

Jens Mazei,Cort W Rudolph,Hannes Zacher,Joachim Hüffmeier

{"title":"Do not put all of your eggs in one basket: Multiverse analysis in applied psychology.","authors":"Jens Mazei,Cort W Rudolph,Hannes Zacher,Joachim Hüffmeier","doi":"10.1037/apl0001291","DOIUrl":null,"url":null,"abstract":"A multiverse analysis allows researchers to systematically evaluate the support for a hypothesis across a range of sensible ways in which data can be prepared for statistical analysis and/or be analyzed. Accordingly, multiverse analysis provides insights into the relevance of different approaches to, for instance, dealing with outliers or attrition, creating scales, or using different measures for the same construct. The goal of this article is to illustrate the usefulness of multiverse analysis for research in applied psychology and to guide researchers in conducting a multiverse analysis. To do so, we provide a detailed process model of the typical stages involved in conducting a multiverse analysis (along with a shortened version depicting multiverse analysis \"at a glance\"), as well as a designated, corresponding preregistration template for multiverse analysis. To showcase the merits of a multiverse analysis, we also evaluate two exemplary hypotheses regarding employees' experience of commuting to and from work. We observed that the results of these hypothesis tests varied strongly depending on how common decisions were made. As such, multiverse analysis represents an important tool for exploring the robustness of knowledge at the level of individual studies, even before a replication is conducted. Hence, multiverse analysis can strengthen the transparency and openness of empirical work. (PsycInfo Database Record (c) 2025 APA, all rights reserved).","PeriodicalId":15135,"journal":{"name":"Journal of Applied Psychology","volume":"2 1","pages":""},"PeriodicalIF":9.4000,"publicationDate":"2025-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Applied Psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/apl0001291","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MANAGEMENT","Score":null,"Total":0}

引用次数: 0

Abstract

A multiverse analysis allows researchers to systematically evaluate the support for a hypothesis across a range of sensible ways in which data can be prepared for statistical analysis and/or be analyzed. Accordingly, multiverse analysis provides insights into the relevance of different approaches to, for instance, dealing with outliers or attrition, creating scales, or using different measures for the same construct. The goal of this article is to illustrate the usefulness of multiverse analysis for research in applied psychology and to guide researchers in conducting a multiverse analysis. To do so, we provide a detailed process model of the typical stages involved in conducting a multiverse analysis (along with a shortened version depicting multiverse analysis "at a glance"), as well as a designated, corresponding preregistration template for multiverse analysis. To showcase the merits of a multiverse analysis, we also evaluate two exemplary hypotheses regarding employees' experience of commuting to and from work. We observed that the results of these hypothesis tests varied strongly depending on how common decisions were made. As such, multiverse analysis represents an important tool for exploring the robustness of knowledge at the level of individual studies, even before a replication is conducted. Hence, multiverse analysis can strengthen the transparency and openness of empirical work. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

查看原文本刊更多论文

不要把所有的鸡蛋放在一个篮子里：应用心理学中的多元宇宙分析。

多元宇宙分析允许研究人员通过一系列合理的方法系统地评估对假设的支持，这些方法可以为统计分析和/或分析数据做好准备。因此，多元宇宙分析提供了对不同方法相关性的见解，例如，处理异常值或损耗，创建尺度，或对同一结构使用不同的测量方法。本文的目的是说明多元宇宙分析对应用心理学研究的有用性，并指导研究人员进行多元宇宙分析。为此，我们提供了进行多元宇宙分析所涉及的典型阶段的详细过程模型（以及描述多元宇宙分析“一目了然”的简短版本），以及指定的、相应的多元宇宙分析预注册模板。为了展示多元宇宙分析的优点，我们还评估了关于员工上下班通勤经验的两个示例假设。我们观察到，这些假设检验的结果在很大程度上取决于共同决策的制定方式。因此，多元宇宙分析是在个体研究层面探索知识稳健性的重要工具，甚至在进行复制之前也是如此。因此，多元宇宙分析可以增强实证工作的透明度和开放性。（PsycInfo Database Record (c) 2025 APA，版权所有）。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Applied Psychology Multiple-

CiteScore

17.60

自引率

6.10%

发文量

175

期刊介绍： The Journal of Applied Psychology® focuses on publishing original investigations that contribute new knowledge and understanding to fields of applied psychology (excluding clinical and applied experimental or human factors, which are better suited for other APA journals). The journal primarily considers empirical and theoretical investigations that enhance understanding of cognitive, motivational, affective, and behavioral psychological phenomena in work and organizational settings. These phenomena can occur at individual, group, organizational, or cultural levels, and in various work settings such as business, education, training, health, service, government, or military institutions. The journal welcomes submissions from both public and private sector organizations, for-profit or nonprofit. It publishes several types of articles, including: 1.Rigorously conducted empirical investigations that expand conceptual understanding (original investigations or meta-analyses). 2.Theory development articles and integrative conceptual reviews that synthesize literature and generate new theories on psychological phenomena to stimulate novel research. 3.Rigorously conducted qualitative research on phenomena that are challenging to capture with quantitative methods or require inductive theory building.