Including conference abstracts rarely changed systematic review conclusions: a case study from a living network meta-analysis of COVID-19 treatments

IF 5.2 2区医学 Q1 HEALTH CARE SCIENCES & SERVICES

Journal of Clinical Epidemiology Pub Date : 2025-08-18 DOI:10.1016/j.jclinepi.2025.111931

Maria Jose Oliveros , Sara Ibrahim , Gonzalo Bravo-Soto , Natalia Chahin-Inostroza , Carlos Zaror , Constanza Ulloa-Lopez , Álvaro Sanhueza , Pamela Seron , Dena Zeraatkar , Gordon Guyatt , Nancy Santesso , Romina Brignardello-Petersen

{"title":"Including conference abstracts rarely changed systematic review conclusions: a case study from a living network meta-analysis of COVID-19 treatments","authors":"Maria Jose Oliveros , Sara Ibrahim , Gonzalo Bravo-Soto , Natalia Chahin-Inostroza , Carlos Zaror , Constanza Ulloa-Lopez , Álvaro Sanhueza , Pamela Seron , Dena Zeraatkar , Gordon Guyatt , Nancy Santesso , Romina Brignardello-Petersen","doi":"10.1016/j.jclinepi.2025.111931","DOIUrl":null,"url":null,"abstract":"<div><h3>Background and Objectives</h3><div>Including conference abstracts (CAs) in systematic reviews (SRs) helps reduce publication bias but raises concerns about reporting quality and reliability. While discrepancies with full publications are known, excluding CAs may overlook relevant early evidence. To evaluate the reporting quality of CAs, their consistency with full-text publications, and the impact of including them on effect estimates and the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework in a living systematic review and network meta-analysis (SRNMA) of COVID-19 drug treatments.</div></div><div><h3>Study Design and Setting</h3><div>We conducted a retrospective methodological study of all CAs included in the COVID-19 SRNMA until May 19, 2024. We assessed trial characteristics, reporting quality using Consolidated Standards of Reporting Trials (CONSORT)-A, and consistency with full-text publications. We also compared meta-analyses with and without CAs at predefined time points for mortality and hospital length of stay, evaluating changes in effect and GRADE domains.</div></div><div><h3>Results</h3><div>We included 105 CAs; 53% (56/105) were linked to a full publication. Only 7% met high reporting standards. Average consistency with full-text publications across key methodological items was 67.6%, often due to missing details in both sources. CAs enabled meta-analyses that would not have been possible at 14% of time points. Their inclusion did not affect conclusions when using the null threshold but changed the effect estimate in 55.6% and imprecision ratings in 16% of cases when using minimally important differences (MIDs). In a few instances, CAs also influenced risk of bias and inconsistency assessments.</div></div><div><h3>Conclusion</h3><div>CAs can fill evidence gaps when data are limited or emerging. Although they rarely change conclusions based on the null threshold, their inclusion has a greater impact when using MID. Reviewers should assess their inclusion case by case and promote better reporting practices to enhance their contribution to SRs.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"187 ","pages":"Article 111931"},"PeriodicalIF":5.2000,"publicationDate":"2025-08-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Clinical Epidemiology","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0895435625002641","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}

引用次数: 0

Abstract

Background and Objectives

Including conference abstracts (CAs) in systematic reviews (SRs) helps reduce publication bias but raises concerns about reporting quality and reliability. While discrepancies with full publications are known, excluding CAs may overlook relevant early evidence. To evaluate the reporting quality of CAs, their consistency with full-text publications, and the impact of including them on effect estimates and the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework in a living systematic review and network meta-analysis (SRNMA) of COVID-19 drug treatments.

Study Design and Setting

We conducted a retrospective methodological study of all CAs included in the COVID-19 SRNMA until May 19, 2024. We assessed trial characteristics, reporting quality using Consolidated Standards of Reporting Trials (CONSORT)-A, and consistency with full-text publications. We also compared meta-analyses with and without CAs at predefined time points for mortality and hospital length of stay, evaluating changes in effect and GRADE domains.

Results

We included 105 CAs; 53% (56/105) were linked to a full publication. Only 7% met high reporting standards. Average consistency with full-text publications across key methodological items was 67.6%, often due to missing details in both sources. CAs enabled meta-analyses that would not have been possible at 14% of time points. Their inclusion did not affect conclusions when using the null threshold but changed the effect estimate in 55.6% and imprecision ratings in 16% of cases when using minimally important differences (MIDs). In a few instances, CAs also influenced risk of bias and inconsistency assessments.

Conclusion

CAs can fill evidence gaps when data are limited or emerging. Although they rarely change conclusions based on the null threshold, their inclusion has a greater impact when using MID. Reviewers should assess their inclusion case by case and promote better reporting practices to enhance their contribution to SRs.

查看原文本刊更多论文

包括会议摘要很少改变系统评价结论：来自COVID-19治疗的生活网络meta分析的案例研究。

背景：将会议摘要（CAs）纳入系统评价（SRs）有助于减少发表偏倚，但也会引起对报告质量和可靠性的担忧。虽然已知与完整出版物的差异，但排除ca可能会忽略相关的早期证据。目的：评价CAs的报告质量、与全文出版物的一致性，以及纳入CAs对COVID-19药物治疗活体系统评价和网络meta分析（SRNMA）中推荐、评估、发展和评价（GRADE）框架的效果估计和分级的影响。研究设计和环境：我们对纳入COVID-19 SRNMA的所有ca进行了回顾性方法学研究，直至2024年5月19日。我们评估了试验特征、使用concur - a的报告质量以及与全文出版物的一致性。我们还比较了在预定的死亡率和住院时间点上有和没有ca的meta分析，评估了效果和GRADE域的变化。结果：我们纳入了105例CAs；53%（56/105）被链接到完整的出版物。只有7%达到了高报告标准。关键方法学项目与全文出版物的平均一致性为67.6%，这通常是由于两个来源都缺少细节。在14%的时间点上，ca使meta分析成为可能。当使用零阈值时，它们的纳入不影响结论，但当使用最小重要差异时，改变了55.6%的效果估计和16%的不精确评级。在少数情况下，CAs还会影响偏倚和不一致评估的风险。结论：当数据有限或新出现时，ca可以填补证据空白。虽然他们很少根据零阈值改变结论，但在使用MID时，他们的纳入有更大的影响。审稿人应逐案评估他们的纳入情况，并促进更好的报告实践，以提高他们对sr的贡献。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Clinical Epidemiology 医学-公共卫生、环境卫生与职业卫生

CiteScore

12.00

自引率

6.90%

发文量

320

审稿时长

44 days

期刊介绍： The Journal of Clinical Epidemiology strives to enhance the quality of clinical and patient-oriented healthcare research by advancing and applying innovative methods in conducting, presenting, synthesizing, disseminating, and translating research results into optimal clinical practice. Special emphasis is placed on training new generations of scientists and clinical practice leaders.