Cochrane Evidence Synthesis and Methods最新文献

{"title":"Using GPT-4 for Title and Abstract Screening in a Literature Review of Public Policies: A Feasibility Study","authors":"Max Rubinstein,&nbsp;Sean Grant,&nbsp;Beth Ann Griffin,&nbsp;Seema Choksy Pessar,&nbsp;Bradley D. Stein","doi":"10.1002/cesm.70031","DOIUrl":"https://doi.org/10.1002/cesm.70031","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>We describe the first known use of large language models (LLMs) to screen titles and abstracts in a review of public policy literature. Our objective was to assess the percentage of articles GPT-4 recommended for exclusion that should have been included (“false exclusion rate”).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We used GPT-4 to exclude articles from a database for a literature review of quantitative evaluations of federal and state policies addressing the opioid crisis. We exported our bibliographic database to a CSV file containing titles, abstracts, and keywords and asked GPT-4 to recommend whether to exclude each article. We conducted a preliminary testing of these recommendations using a subset of articles and a final test on a sample of the entire database. We designated a false exclusion rate of 10% as an adequate performance threshold.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>GPT-4 recommended excluding 41,742 of the 43,480 articles (96%) containing an abstract. Our preliminary test identified only one false exclusion; our final test identified no false exclusions, yielding an estimated false exclusion rate of 0.00 [0.00, 0.05]. Fewer than 1%—417 of the 41,742 articles—were incorrectly excluded. After manually assessing the eligibility of all remaining articles, we identified 608 of the 1738 articles that GPT-4 did not exclude: 65% of the articles recommended for inclusion should have been excluded.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Discussion/Conclusions</h3>\u0000 \u0000 <p>GPT-4 performed well at recommending articles to exclude from our literature review, resulting in substantial time and cost savings. A key limitation is that we did not use GPT-4 to determine inclusions, nor did our model perform well on this task. However, GPT-4 dramatically reduced the number of articles requiring review. Systematic reviewers should conduct performance evaluations to ensure that an LLM meets a minimally acceptable quality standard before relying on its recommendations.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70031","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144108810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Artificial Intelligence and Machine Learning to Improve Evidence Synthesis Production Efficiency: An Observational Study of Resource Use and Time-to-Completion","authors":"Christopher James Rose,&nbsp;Jose Francisco Meneses-Echavez,&nbsp;Ashley Elizabeth Muller,&nbsp;Rigmor C. Berg,&nbsp;Tiril C. Borge,&nbsp;Patricia Sofia Jacobsen Jardim,&nbsp;Chris Cooper","doi":"10.1002/cesm.70030","DOIUrl":"https://doi.org/10.1002/cesm.70030","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Evidence syntheses are crucial in healthcare and elsewhere but are resource-intensive, often taking years to produce. Artificial intelligence and machine learning (AI/ML) tools may improve production efficiency in certain review phases, but little is known about their impact on entire reviews.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We performed prespecified analyses of a convenience sample of eligible healthcare- or welfare-related reviews commissioned at the Norwegian Institute of Public Health between August 1 2020 (first commission to use AI/ML) and January 31 2023 (administrative cut-off). The main exposures were AI/ML use following an internal support team's recommendation versus no use. Ranking (e.g., priority screening), classification (e.g., study design), clustering (e.g., documents), and bibliometric analysis (e.g., OpenAlex) tools were included, but we did not include or exclude specific tools. Generative AI tools were not widely available during the study period. The outcomes were resources (person-hours) and time from commission to completion (approval for delivery, including peer review; weeks). Analyses accounted for nonrandomized assignment and censored outcomes (reviews ongoing at cut-off). Researchers classifying exposures were blinded to outcomes. The statistician was blinded to exposure.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Among 39 reviews, 7 (18%) were health technology assessments versus systematic reviews, 19 (49%) focused on healthcare versus welfare, 18 (46%) planned meta-analysis, and 3 (8%) were ongoing at cut-off. AI/ML tools were used in 27 (69%) reviews. Reviews that used AI/ML as recommended used more resources (mean 667 vs. 291 person-hours) but were completed slightly faster (27.6 vs. 28.2 weeks). These differences were not statistically significant (relative resource use 3.71; 95% CI: 0.36–37.95; <i>p</i> = 0.269; relative time-to-completion: 0.92; 95% CI: 0.53–1.58; <i>p</i> = 0.753).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Associations between AI/ML use and the outcomes remains uncertain. Multicenter studies or meta-analyses may be needed to determine if these tools meaningfully reduce resource use and time to produce evidence syntheses.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70030","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144085014","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Information Practice as Dialogue: The Case for Collaboration in Evidence Searching and Finding for More Complex Reviews","authors":"Parkhill Anne,&nbsp;Merner Bronwen,&nbsp;Ryan Rebecca","doi":"10.1002/cesm.70029","DOIUrl":"https://doi.org/10.1002/cesm.70029","url":null,"abstract":"&lt;p&gt;Cochrane Consumers and Communication Group's (CCC) approach to evidence searching has evolved over time in the context of Cochrane's rigorous methodological advice [&lt;span&gt;1, 2&lt;/span&gt;]. CCC is a Cochrane review group responsible for coordinating the preparation and publication of evidence syntheses that affect the way people interact with healthcare professionals, services and researchers. CCC includes a highly skilled Information Specialist who collaborates with CCC author teams to design a rigorous search strategy to gather evidence to answer the review question. In this commentary, we discuss the transformation of the information practice of searching in CCC from being a largely technical exercise conducted solely by the Information Specialist to a collaborative dialogue between the Information Specialist and author teams.&lt;/p&gt;&lt;p&gt;A key reason for the transformation in our search methods has been that CCC reviews tend to be complex, with review questions that are generally not as easily answered as clinically focused reviews. Our research, and information practice specifically, is contextualized and guided by a three-way dynamic of patient preferences and experiences, research evidence, and professional expertize. The reviews are rigorous in their examination of evidence on people's healthcare interactions, including how people self-manage health and disease, understand screening, health and treatment, and negotiate and share decisions with healthcare professionals within systems and different settings. However, interventions to change behaviors, to educate, support and up-skill people to participate actively in their healthcare, are often complex, multifaceted and their effects evaluated via multiple diverse outcomes [&lt;span&gt;3&lt;/span&gt;]. This complexity necessarily shapes our methods of information practice.&lt;/p&gt;&lt;p&gt;Early in the life of CCC and for many years, we viewed searching as a largely solitary technical exercise performed by a skilled Information Specialist following conventional, rigorous Cochrane search methods. Often this required labor-intensive search development, resulting in delays for search results and an excessive screening obligation (e.g., some review questions resulted in authors needing to screen more than 25,000 search results). As volume and complexity of literature in the health communication area increased, we moved towards search strategies developed with practicalities of reference screening in mind [&lt;span&gt;4, 5&lt;/span&gt;]. We have since developed transparent and pragmatic search strategies by means of embedded and open dialogue [&lt;span&gt;6&lt;/span&gt;] with authors. In the context of increasing topic complexity and rigorous information searching, this approach maximizes identification of relevant references while avoiding unmanageable reference numbers for screening.&lt;/p&gt;&lt;p&gt;In this commentary, we explore CCC's approach to searching and its evolution over time in the context of Cochrane's rigorous methodological advice. We illustrat","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-04-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70029","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143883954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessing the reporting quality of published qualitative evidence syntheses in the cochrane library","authors":"Martina Giltenane,&nbsp;Aoife O'Mahony,&nbsp;Mayara S. Bianchim,&nbsp;Andrew Booth,&nbsp;Angela Harden,&nbsp;Catherine Houghton,&nbsp;Emma F. France,&nbsp;Heather Ames,&nbsp;Kate Flemming,&nbsp;Katy Sutcliffe,&nbsp;Ruth Garside,&nbsp;Tomas Pantoja,&nbsp;Jane Noyes","doi":"10.1002/cesm.70023","DOIUrl":"https://doi.org/10.1002/cesm.70023","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Over ten years since the first qualitative evidence synthesis (QES) was published in the Cochrane Library, QES and mixed-methods reviews (MMR) with a qualitative component have become increasingly common and influential in healthcare research and policy development. The quality of such reviews and the completeness with which they are reported is therefore of paramount importance.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Aim</h3>\u0000 \u0000 <p>This review aimed to assess the reporting quality of published QESs and MMRs with a qualitative component in the Cochrane Library.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>All published QESs and MMRs were identified from the Cochrane Library. A bespoke framework developed by key international experts based on the Effective Practice and Organisation of Care (EPOC), Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) and meta-ethnography reporting guidance (eMERGe) was used to code the quality of reporting of QESs and MMRs.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Thirty-one reviews were identified, including 11 MMRs. The reporting quality of the QESs and MMRs published by Cochrane varied considerably. Based on the criteria within our framework, just over a quarter (8, 26%) were considered to meet satisfactory reporting standards, 10 (32%) could have provided clearer or more detailed descriptions in their reporting, just over a quarter (8, 26%) provided poor quality or insufficient descriptions and five (16%) omitted descriptions relevant to our framework.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>This assessment offers important insights into the reporting practices prevalent in these review types. Methodology and reporting have changed considerably over time. Earlier QES have not necessarily omitted important reporting components, but rather our understanding of what should be completed and reported has grown considerably. The variability in reporting quality within QESs and MMRs underscores the need to develop Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) specifically for QES.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70023","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143836206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Should we adopt the case report format to report challenges in complicated evidence synthesis? A proposal and illustration of a case report of a complex search strategy for humanitarian interventions","authors":"Chris Cooper,&nbsp;Zahra Premji,&nbsp;Cem Yavuz,&nbsp;Mark Engelbert","doi":"10.1002/cesm.70021","DOIUrl":"https://doi.org/10.1002/cesm.70021","url":null,"abstract":"<p>Case reports represent a form of evidence in medicine which detail an unusual or novel clinical case in a short, published report, disseminated for the attention of clinical staff. This form of report is not common outside of clinical practice. We question if the adoption of the ‘case report’ might also be useful in evidence synthesis. This where the case represents a challenge in undertaking evidence synthesis and the report details not only the resolution but also shows the working to resolve the challenge. Our rationale is that methodological responses to problems arising in complicated evidence synthesis often go unreported. The risk is that lessons learned in developing evidence synthesis are lost if not recorded. This represents a form of research waste. We suggest that the adoption of the case report format might represent the opportunity to highlight not only a challenge (the case) but a worked example of a possible solution (the report). These case reports would represent a resting place for the case, with notes left behind for future researchers to follow. We provide an example of a case report: a complicated search strategy developed to inform an evidence gap map on the effects of interventions in humanitarian settings on food security outcomes in low and middle-income countries and specific high-income countries. Our report details the solution that we developed (the search strategy). We also illustrate how we conceptualised the search, and the approaches that we tested but rejected, and the ideas that we pursued.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70021","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143826729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A New Process Model of Study Identification Specific to the Identification of Randomised Studies for Systematic Reviews of Medical Interventions","authors":"Chris Cooper,&nbsp;Zahra Premji,&nbsp;Christine Worsley,&nbsp;Eve Tomlinson,&nbsp;Sarah Dawson,&nbsp;Emma Prentice","doi":"10.1002/cesm.70026","DOIUrl":"https://doi.org/10.1002/cesm.70026","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Recent work has illustrated that the same process of study identification is used in systematic reviews irrespective of the studies or data needs required for synthesis. We question if different review types should have their own specific models of study identification, to ensure the appropriate and timely identification of studies/study reports and to minimise research waste.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Objective</h3>\u0000 \u0000 <p>In this paper, we aim to:</p>\u0000 \u0000 <p>1. illustrate and report a new process model to identify randomised studies for systematic reviews of medical interventions; and</p>\u0000 \u0000 <p>2. situate the model in context of current practice using a worked example from a recent systematic review.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Method</h3>\u0000 \u0000 <p>Our model splits the identification of studies from the identification of study reports by searching in distinct phases. It begins with searches of trials registry resources to identify studies, followed by searches of bibliographic databases to identify study reports or unregistered studies. Supplementary search methods are then used to identify unpublished studies. The model includes the possibility of secondary searches, and we consider the role of update searches.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>A case study illustrates the application of the method alongside operational guidance.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70026","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143826730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multiple Long-Term Conditions, Co-Long-Term Conditions and Polyvascular Disease: Considerations for Evidence Synthesis and Meta-Analyses","authors":"Gillian Mead,&nbsp;Alex Todhunter-Brown,&nbsp;Ukachukwu Abaraogu,&nbsp;Amanda Barugh,&nbsp;Arohi Chauhan,&nbsp;Juan Erviti Lopez,&nbsp;Valery Feigin,&nbsp;Jaya Singh Kshatri,&nbsp;Atsushi Mizuno,&nbsp;Sanghamitra Pati,&nbsp;Jackie Price,&nbsp;Rui Providência,&nbsp;Gerry Stansby,&nbsp;Rod Taylor,&nbsp;David J. Williams,&nbsp;James M. Wright,&nbsp;Simiao Wu,&nbsp;Leon Flicker","doi":"10.1002/cesm.70027","DOIUrl":"https://doi.org/10.1002/cesm.70027","url":null,"abstract":"<p>Cochrane's scientific strategy for 2025 to 2030 has four research priorities, including improving the lives of people living with multiple chronic conditions. The purpose of this article written by the Cochrane Thematic Group in Heart, Stroke and Circulation is to explore considerations around multiple chronic conditions (also referred to as ‘multiple long-term conditions’ i.e. two or more long-term conditions) in systematic reviews. Rather than using the term ‘comorbidity’, we introduce a new term ‘co-long-term conditions’. We also explore how to define ‘polyvascular disease’. We suggest that review authors consider co-long-term conditions and multiple long-term conditions in their reviews e.g. extract data about how primary studies address co-long-term conditions, perform subgroup analyses according to presence or not of co-long-term conditions, and include a section in the discussion about how well participants with co-long-term conditions were represented in the primary studies. This is especially pertinent for reviews addressing heart, circulatory or stroke disease, and polyvascular disease.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70027","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143769980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Cochrane's COVID-19 Living Systematic Reviews: A Mixed-Methods Study of Their Conduct, Reporting and Currency","authors":"Kevindu De Silva,&nbsp;Tari Turner,&nbsp;Steve McDonald","doi":"10.1002/cesm.70024","DOIUrl":"https://doi.org/10.1002/cesm.70024","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Living systematic reviews (LSRs) should provide up-to-date evidence for priority questions where the evidence may be uncertain and fast-moving. LSRs featured prominently during COVID-19 and formed part of Cochrane's response to the pandemic. We conducted a mixed-methods study to describe the characteristics of Cochrane's COVID-19 living reviews, determine the currency of the included evidence, and evaluate authors' experiences in conducting and publishing these reviews.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We identified living reviews of COVID-19 from the <i>Cochrane Database of Systematic Reviews</i> and extracted data on the number of versions published and publication timelines. We assessed the currency of evidence by comparing studies included in the reviews against a comprehensive list of studies maintained for the Australian living guidelines for COVID-19. The qualitative component involved semi-structured interviews with review authors to identify the barriers and enablers to conducting, reporting and publishing living reviews.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Findings</h3>\u0000 \u0000 <p>Cochrane published 25 COVID-19 living systematic reviews. Half of these reviews had not been updated when assessed in June 2023 and only four had been updated more than once. A total of 118 studies were included in the living reviews. We estimated that an additional 119 studies were available and potentially relevant for inclusion. Interviews with six authors indicated that publication timelines were reduced by editorial delays, loss of funding, waning commitment, and the burden of screening search results. An inability to communicate the living status of reviews in the Cochrane Library was a common frustration for many authors. Although authors felt the conclusions of their reviews were still current, only one living review communicated its updated status and made new evidence accessible after the review was published.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>Maintaining and communicating the currency of Cochrane's COVID-19 living systematic reviews was not feasible for many author teams because of author-side, editorial and platform barriers.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70024","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143717433","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Certainty of evidence assessment in high-impact medical journals: A meta-epidemiological survey","authors":"Madelin R. Siedler,&nbsp;Neha Tangri,&nbsp;Leena AlShenaiber,&nbsp;Tejanth Pasumarthi,&nbsp;Faisal Shaukat Ali,&nbsp;Volf Gaby,&nbsp;Katie N. Harris,&nbsp;Yngve Falck-Ytter,&nbsp;Reem A. Mustafa,&nbsp;Shahnaz Sultan,&nbsp;Philipp Dahm,&nbsp;M. Hassan Murad,&nbsp;Rebecca L. Morgan","doi":"10.1002/cesm.70014","DOIUrl":"https://doi.org/10.1002/cesm.70014","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>While certainty of evidence assessment is key to a rigorous and transparent systematic review, it is unknown how – and how frequently – it is assessed in systematic reviews. The objective of this study was to examine the prevalence and approaches used for certainty of evidence assessment in systematic reviews published in high-impact medicine journals over the past 11 years.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>A PubMed search and hand-searching of relevant journal websites identified systematic reviews published between 24 January 2013 and 23 January 2024 in any of the ten highest-impact journals in the General and Internal Medicine category of the Journal Citation Report. Two reviewers independently selected any systematic review related to health outcomes assessing certainty of evidence using any method. We extracted data related to review characteristics, certainty of evidence and risk of bias/methodological quality assessment frameworks, and reported consideration of certainty of evidence domains. Logistic regression examined year of publication to determine whether the prevalence of certainty of evidence assessment changed over time.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Of 1,023 included reviews, 346 (33.8%) assessed certainty of evidence. Prevalence of certainty of evidence assessment increased over time (0.16 ± 0.2; <i>p</i> &lt; .001). Most (89.3%) of reviews used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to assess certainty of evidence.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Only one in three systematic reviews published in the highest-impact medical journals over the past 11 years assessed certainty of evidence, though prevalence increased over time. The use of specific domains within each certainty of evidence framework was not clearly described in all reviews.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70014","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143688888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Coproducing a Cochrane Qualitative Evidence Synthesis: Process, Outcomes, and Reflections on Power","authors":"Bronwen Merner,&nbsp;Rebecca Ryan","doi":"10.1002/cesm.70025","DOIUrl":"https://doi.org/10.1002/cesm.70025","url":null,"abstract":"<p>Reflecting a broader movement toward knowledge democratization, coproducing Cochrane evidence with interest holders outside universities is increasingly encouraged. However, only limited research exists on the approaches used to coproduce Cochrane reviews. Furthermore, the outcomes of coproduction are rarely described. In this commentary, we aim to address these gaps by describing the process and outcomes of coproduction used in a recently published Cochrane qualitative evidence synthesis (QES). We also reflect on power imbalances in our coproduction approach and how these could be minimized in future review coproduction activities.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70025","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143645809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}