Assessment of inverse publication bias in safety outcomes: an empirical analysis.

IF 7 1区 医学 Q1 MEDICINE, GENERAL & INTERNAL
Xing Xing, Jianan Zhu, Linyu Shi, Chang Xu, Lifeng Lin
{"title":"Assessment of inverse publication bias in safety outcomes: an empirical analysis.","authors":"Xing Xing, Jianan Zhu, Linyu Shi, Chang Xu, Lifeng Lin","doi":"10.1186/s12916-024-03707-2","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The aims of this study were to assess the presence of inverse publication bias (IPB) in adverse events, evaluate the performance of visual examination, and explore the impact of considering effect direction in statistical tests for such assessments.</p><p><strong>Methods: </strong>We conducted a cross-sectional study using the SMART Safety, the largest dataset for evidence synthesis of adverse events. The visual assessment was performed using contour-enhanced funnel plots, trim-and-fill funnel plots, and sample-size-based funnel plots. Two authors conducted visual assessments of these plots independently, and their agreements were quantified by the kappa statistics. Additionally, IPB was quantitatively assessed using both the one- and two-sided Egger's and Peters' tests.</p><p><strong>Results: </strong>In the SMART Safety dataset, we identified 277 main meta-analyses of safety outcomes with at least 10 individual estimates after dropping missing data. We found that about 13.7-16.2% of meta-analyses exhibited IPB according to the one-sided test results. The kappa statistics for the visual assessments roughly ranged from 0.3 to 0.5, indicating fair to moderate agreement. Using the one-sided Egger's test, 57 out of 72 (79.2%) meta-analyses that initially showed significant IPB in the two-sided test changed to non-significant, while the remaining 15 (20.8%) meta-analyses changed from non-significant to significant.</p><p><strong>Conclusions: </strong>Our findings provide supporting evidence of IPB in the SMART Safety dataset of adverse events. They also suggest the importance of researchers carefully accounting for the direction of statistical tests for IPB, as well as the challenges of assessing IPB using statistical methods, especially considering that the number of studies is typically small. Qualitative assessments may be a necessary supplement to gain a more comprehensive understanding of IPB.</p>","PeriodicalId":9188,"journal":{"name":"BMC Medicine","volume":null,"pages":null},"PeriodicalIF":7.0000,"publicationDate":"2024-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11515227/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12916-024-03707-2","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
引用次数: 0

Abstract

Background: The aims of this study were to assess the presence of inverse publication bias (IPB) in adverse events, evaluate the performance of visual examination, and explore the impact of considering effect direction in statistical tests for such assessments.

Methods: We conducted a cross-sectional study using the SMART Safety, the largest dataset for evidence synthesis of adverse events. The visual assessment was performed using contour-enhanced funnel plots, trim-and-fill funnel plots, and sample-size-based funnel plots. Two authors conducted visual assessments of these plots independently, and their agreements were quantified by the kappa statistics. Additionally, IPB was quantitatively assessed using both the one- and two-sided Egger's and Peters' tests.

Results: In the SMART Safety dataset, we identified 277 main meta-analyses of safety outcomes with at least 10 individual estimates after dropping missing data. We found that about 13.7-16.2% of meta-analyses exhibited IPB according to the one-sided test results. The kappa statistics for the visual assessments roughly ranged from 0.3 to 0.5, indicating fair to moderate agreement. Using the one-sided Egger's test, 57 out of 72 (79.2%) meta-analyses that initially showed significant IPB in the two-sided test changed to non-significant, while the remaining 15 (20.8%) meta-analyses changed from non-significant to significant.

Conclusions: Our findings provide supporting evidence of IPB in the SMART Safety dataset of adverse events. They also suggest the importance of researchers carefully accounting for the direction of statistical tests for IPB, as well as the challenges of assessing IPB using statistical methods, especially considering that the number of studies is typically small. Qualitative assessments may be a necessary supplement to gain a more comprehensive understanding of IPB.

安全结果的反向发表偏差评估:实证分析。
研究背景本研究的目的是评估不良事件中是否存在反向发表偏倚(IPB),评价目测检查的性能,并探讨在此类评估的统计检验中考虑效应方向的影响:我们利用最大的不良事件证据综合数据集 SMART Safety 开展了一项横断面研究。使用轮廓增强漏斗图、修剪填充漏斗图和基于样本大小的漏斗图进行视觉评估。两位作者独立对这些漏斗图进行了视觉评估,并通过卡帕统计对他们的一致性进行了量化。此外,还使用单侧和双侧 Egger 检验和 Peters 检验对 IPB 进行了定量评估:在 SMART 安全性数据集中,我们发现了 277 项主要的安全性结果荟萃分析,在剔除缺失数据后,至少有 10 个单独的估计值。我们发现,根据单侧检验结果,约有 13.7%-16.2%的荟萃分析显示出 IPB。视觉评估的 kappa 统计量大致在 0.3 到 0.5 之间,表明一致性尚可。使用单侧 Egger 检验,72 项元分析中有 57 项(79.2%)最初在双侧检验中显示出显著的 IPB,但后来转为不显著,而其余 15 项元分析(20.8%)则由不显著转为显著:我们的研究结果为 SMART 安全性不良事件数据集中的 IPB 提供了支持性证据。这些研究结果还表明,研究人员必须仔细考虑 IPB 统计检验的方向,以及使用统计方法评估 IPB 所面临的挑战,尤其是考虑到研究的数量通常较少。为了更全面地了解 IPB,定性评估可能是必要的补充。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
BMC Medicine
BMC Medicine 医学-医学:内科
CiteScore
13.10
自引率
1.10%
发文量
435
审稿时长
4-8 weeks
期刊介绍: BMC Medicine is an open access, transparent peer-reviewed general medical journal. It is the flagship journal of the BMC series and publishes outstanding and influential research in various areas including clinical practice, translational medicine, medical and health advances, public health, global health, policy, and general topics of interest to the biomedical and sociomedical professional communities. In addition to research articles, the journal also publishes stimulating debates, reviews, unique forum articles, and concise tutorials. All articles published in BMC Medicine are included in various databases such as Biological Abstracts, BIOSIS, CAS, Citebase, Current contents, DOAJ, Embase, MEDLINE, PubMed, Science Citation Index Expanded, OAIster, SCImago, Scopus, SOCOLAR, and Zetoc.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信