Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

IF 5 2区生物学 Q1 MATHEMATICAL & COMPUTATIONAL BIOLOGY

Research Synthesis Methods Pub Date : 2024-02-07 DOI:10.1002/jrsm.1703

František Bartoš, Maximilian Maier, Eric-Jan Wagenmakers, Franziska Nippold, Hristos Doucouliagos, John P. A. Ioannidis, Willem M. Otte, Martina Sladekova, Teshome K. Deresssa, Stephan B. Bruns, Daniele Fanelli, T. D. Stanley

{"title":"Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics","authors":"František Bartoš, Maximilian Maier, Eric-Jan Wagenmakers, Franziska Nippold, Hristos Doucouliagos, John P. A. Ioannidis, Willem M. Otte, Martina Sladekova, Teshome K. Deresssa, Stephan B. Bruns, Daniele Fanelli, T. D. Stanley","doi":"10.1002/jrsm.1703","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <p>Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from <i>d</i> = 0.20 to <i>d</i> = 0.07 in economics, from <i>d</i> = 0.37 to <i>d</i> = 0.26 in psychology, from <i>d</i> = 0.62 to <i>d</i> = 0.43 in environmental sciences, and from <i>d</i> = 0.24 to <i>d</i> = 0.13 in medicine.</p>\n </section>\n </div>","PeriodicalId":226,"journal":{"name":"Research Synthesis Methods","volume":"15 3","pages":"500-511"},"PeriodicalIF":5.0000,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/jrsm.1703","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Research Synthesis Methods","FirstCategoryId":"99","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1703","RegionNum":2,"RegionCategory":"生物学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICAL & COMPUTATIONAL BIOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Abstract Image

查看原文本刊更多论文

出版选择偏差对医学、环境科学、心理学和经济学荟萃分析的影响。

发表选择偏差破坏了证据的系统积累。为了评估这一问题的严重程度，我们调查了 68,000 多项元分析，其中包含 700,000 多项效应大小估计，这些元分析分别来自医学（67,386/597,699）、环境科学（199/12,707）、心理学（605/23,563）和经济学（327/91,421）。我们的研究结果表明，经济学领域的荟萃分析受发表选择偏倚的影响最严重，环境科学和心理学领域的荟萃分析紧随其后，而医学领域的荟萃分析受影响最小。在对发表选择偏差进行调整后，经济学领域存在效应的概率中位数从 99.9% 降至 29.7%，心理学领域从 98.9% 降至 55.7%，环境科学领域从 99.8% 降至 70.7%，医学领域从 38.0% 降至 29.7%。绝对效应大小的中位数（标准化均值差异）在经济学中从 d = 0.20 降至 d = 0.07，在心理学中从 d = 0.37 降至 d = 0.26，在环境科学中从 d = 0.62 降至 d = 0.43，在医学中从 d = 0.24 降至 d = 0.13。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Research Synthesis Methods MATHEMATICAL & COMPUTATIONAL BIOLOGYMULTID-MULTIDISCIPLINARY SCIENCES

CiteScore

16.90

自引率

3.10%

发文量

期刊介绍： Research Synthesis Methods is a reputable, peer-reviewed journal that focuses on the development and dissemination of methods for conducting systematic research synthesis. Our aim is to advance the knowledge and application of research synthesis methods across various disciplines. Our journal provides a platform for the exchange of ideas and knowledge related to designing, conducting, analyzing, interpreting, reporting, and applying research synthesis. While research synthesis is commonly practiced in the health and social sciences, our journal also welcomes contributions from other fields to enrich the methodologies employed in research synthesis across scientific disciplines. By bridging different disciplines, we aim to foster collaboration and cross-fertilization of ideas, ultimately enhancing the quality and effectiveness of research synthesis methods. Whether you are a researcher, practitioner, or stakeholder involved in research synthesis, our journal strives to offer valuable insights and practical guidance for your work.