Cochrane Evidence Synthesis and Methods最新文献

{"title":"Introducing a Series of Reviews Assessing Engagement in Evidence Syntheses","authors":"Jennifer Petkovic, Joanne Khabsa, Lyubov Lytvyn, Alex Todhunter-Brown, Olivia Magwood, Pauline Campbell, Elie A. Akl, Thomas W. Concannon, Holger Schunemann, Vivian Welch, Peter Tugwell","doi":"10.1002/cesm.70057","DOIUrl":"https://doi.org/10.1002/cesm.70057","url":null,"abstract":"<p>High quality evidence syntheses are used in health decision-making, such as policies, legislation, and clinical recommendations [<span>1</span>]. The usefulness, relevance, meaningfulness, and accessibility of evidence syntheses may be improved when people who are affected by those decisions, called “interest-holders,” are included in the evidence synthesis process [<span>2-4</span>]. This concept of engagement in research is based on the principle that those affected by the health condition under study or the intervention to address it have a moral right to contribute to the decisions about how the research is conducted [<span>3, 5</span>]. While there are increasing expectations from funders regarding the involvement of interest-holders [<span>6</span>], the most effective methods for engaging different interest-holders in evidence syntheses have not been identified [<span>5</span>]. Additionally, while there is some guidance related to engagement in research, it predominantly focuses on patient and public engagement in primary research, not evidence synthesis and there is limited guidance for engaging with other interest-holders [<span>3, 4, 7-9</span>].</p><p>The aim of this paper is to introduce a series of articles about how to successfully engage different interest-holders when conducting evidence syntheses. The series of articles will consider methods used to engage different interest-holders (including who to involve and in what way), barriers and facilitators to engagement, impacts of engagement, management of conflicts of interest, and factors relating to equity.</p><p>This paper presents the shared definitions used across each of the five reviews included in this series. These reviews will inform the development of a guidance checklist and resources for engaging interest-holders through all steps of evidence synthesis. The plan for developing this guidance is described in the project protocol [<span>10</span>].</p><p>“Interest-holders” are groups of people with legitimate interests in the health issue under consideration and whose perspectives and views should be considered when conducting this study [<span>2</span>]. Their interests arise and draw their legitimacy from the fact that these people are responsible for or affected by health- and healthcare-related decisions that can be informed by research evidence. Engagement of interest-holders in evidence syntheses can promote transparency, accountability, trust, and help to ensure that the needs of interest-holders are included. Engagement can improve the translation of evidence into policy and practice [<span>11</span>]. Interest-holders can contribute throughout the steps of evidence synthesis including, for example, refining the research question and suggesting appropriate outcomes, suggesting additional references to consider, and providing context to interpret the evidence.</p><p>This study was conducted by the MuSE Consortium, a group of over 160 individuals from 20 countrie","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70057","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145581023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessing the Feasibility and Acceptability of a Bespoke Large Language Model Pipeline to Extract Data From Different Study Designs for Public Health Evidence Reviews","authors":"Zalaya Simmons,&nbsp;Beti Evans,&nbsp;Tamsyn Harris,&nbsp;Harry Woolnough,&nbsp;Lauren Dunn,&nbsp;Jonathon Fuller,&nbsp;Kerry Cella,&nbsp;Daphne Duval","doi":"10.1002/cesm.70061","DOIUrl":"10.1002/cesm.70061","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Data extraction is a critical but resource-intensive step of the evidence review process. Whilst there is evidence that artificial intelligence (AI) and large language models (LLMs) can improve the efficiency of data extraction from randomized controlled trials, their potential for other study designs is unclear. In this context, this study aimed to evaluate the performance of a bespoke LLM model pipeline (Retrieval-Augmented Generation pipeline utilizing LLaMa 3-70B) to automate data extraction from a range of study designs by assessing the accuracy and reliability of the extractions measured as error types and acceptability.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>Accuracy was assessed by retrospectively comparing the LLM extractions against human extractions from a review previously conducted by the authors. A total of 173 data fields from 24 articles (including experimental, observational, qualitative, and modeling studies) were assessed, of which three were used for prompt engineering. Reliability was assessed by calculating the mean maximum agreement rate (the highest proportion of identical returns from 10 consecutive extractions) for 116 data fields from 16 of the 24 studies. An evaluation framework was developed to assess the accuracy and reliability of LLM outputs measured as error types and acceptability (acceptability was assessed on whether it would be usable in real-world settings if the model acted as one reviewer and a human as a second reviewer).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Of the 173 data fields evaluated for accuracy, 68% were rated by human reviewers as acceptable (consistent with what is deemed to be acceptable data extraction from a human reviewer). However, acceptability ratings varied depending on the data field extracted (33% to 100%), with at least 90% acceptability for “objective,” “setting,” and “study design,” but 54% or less for data fields such as “outcome” and “time period.” For reliability, the mean maximum agreement rate was 0.71 (SD: 0.28), with variation across different data fields.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>This evaluation demonstrates the potential for LLMs, when paired with human quality assurance, to support data extraction in evidence reviews that include a range of study designs. However, further improvements in performance and validation are required before the model can be introduced into review workflows.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12584109/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145454519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Health Equity in Systematic Reviews: A Tutorial—Part 1 Getting Started With Health Equity in Your Review","authors":"Jennifer Petkovic,&nbsp;Jordi Pardo Pardo,&nbsp;Vivian Welch,&nbsp;Omar Dewidar,&nbsp;Lara J. Maxwell,&nbsp;Andrea Darzi,&nbsp;Tamara Lotfi,&nbsp;Lawrence Mbuagbaw,&nbsp;Kevin Pottie,&nbsp;Peter Tugwell","doi":"10.1002/cesm.70055","DOIUrl":"https://doi.org/10.1002/cesm.70055","url":null,"abstract":"<p>This tutorial focuses on how to get started with considering health equity in systematic reviews of interventions. We will explain why health equity should be considered, how to frame your question, and which interest-holders to engage. This is the first tutorial in a series on health equity. The second tutorial focuses on implementing health equity methods in your review.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70055","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145406969","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Harnessing Large-Language Models for Efficient Data Extraction in Systematic Reviews: The Role of Prompt Engineering","authors":"Molly Murton,&nbsp;Ellie Boulton,&nbsp;Shona Cross,&nbsp;Ambar Khan,&nbsp;Swati Kumar,&nbsp;Giuseppina Magri,&nbsp;Charlotte Marris,&nbsp;David Slater,&nbsp;Emma Worthington,&nbsp;Elizabeth Lunn","doi":"10.1002/cesm.70058","DOIUrl":"10.1002/cesm.70058","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>Systematic literature reviews (SLRs) of randomized clinical trials (RCTs) underpin evidence-based medicine but can be limited by the intensive resource demands of data extraction. Recent advances in accessible large-language models (LLMs) hold promise for automating this step, however testing is limited across different outcomes and disease areas.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>This study developed prompt engineering strategies for GPT-4o to extract data from RCTs across three disease areas: non-small cell lung cancer, endometrial cancer and hypertrophic cardiomyopathy. Prompts were iteratively refined during the development phase, then tested on unseen data. Performance was evaluated via comparison to human extraction of the same data, using F1 scores, precision, recall and percentage accuracy.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>The LLM was highly effective for extracting study and baseline characteristics, often equaling human performance, with test F1 scores exceeding 0.85. Complex efficacy and adverse event data proved more challenging, with test F1 scores ranging from 0.22 to 0.50. Transferability of prompts across disease areas was promising but varied, highlighting the need for disease-specific refinement.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Our findings demonstrate the potential of LLMs, guided by rigorous prompt engineering, to augment the SLR process. However, human oversight remains essential, particularly for complex and nuanced data. As these technologies evolve, continued validation of AI tools will be necessary to ensure accuracy and reliability, and safeguarding of the quality of evidence synthesis.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12559671/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145403637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Human-in-the-Loop Artificial Intelligence System for Systematic Literature Review: Methods and Validations for the AutoLit Review Software","authors":"Kevin M. Kallmes,&nbsp;Jade Thurnham,&nbsp;Marius Sauca,&nbsp;Ranita Tarchand,&nbsp;Keith R. Kallmes,&nbsp;Karl J. Holub","doi":"10.1002/cesm.70059","DOIUrl":"https://doi.org/10.1002/cesm.70059","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Introduction</h3>\u0000 \u0000 <p>While artificial intelligence (AI) tools have been utilized for individual stages within the systematic literature review (SLR) process, no tool has previously been shown to support each critical SLR step. In addition, the need for expert oversight has been recognized to ensure the quality of SLR findings. Here, we describe a complete methodology for utilizing our AI SLR tool with human-in-the-loop curation workflows, as well as AI validations, time savings, and approaches to ensure compliance with best review practices.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>SLRs require completing Search, Screening, and Extraction from relevant studies, with meta-analysis and critical appraisal as relevant. We present a full methodological framework for completing SLRs utilizing our AutoLit software (Nested Knowledge). This system integrates AI models into the central steps in SLR: Search strategy generation, Dual Screening of Titles/Abstracts and Full Texts, and Extraction of qualitative and quantitative evidence. The system also offers manual Critical Appraisal and Insight drafting and fully-automated Network Meta-analysis. Validations comparing AI performance to experts are reported, and where relevant, time savings and ‘rapid review’ alternatives to the SLR workflow.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>Search strategy generation with the Smart Search AI can turn a Research Question into full Boolean strings with 76.8% and 79.6% Recall in two validation sets. Supervised machine learning tools can achieve 82–97% Recall in reviewer-level Screening. Population, Interventions/Comparators, and Outcomes (PICOs) extraction achieved F1 of 0.74; accuracy for study type, location, and size were 74%, 78%, and 91%, respectively. Time savings of 50% in Abstract Screening and 70–80% in qualitative extraction were reported. Extraction of user-specified qualitative and quantitative tags and data elements remains exploratory and requires human curation for SLRs.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>AI systems can support high-quality, human-in-the-loop execution of key SLR stages. Transparency, replicability, and expert oversight are central to the use of AI SLR tools.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70059","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145367149","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing Evidence Synthesis Efficiency: Leveraging Large Language Models and Agentic Workflows for Optimized Literature Screening","authors":"Bing Hu,&nbsp;Emmalie Tomini,&nbsp;Tricia Corrin,&nbsp;Kusala Pussegoda,&nbsp;Elias Sandner,&nbsp;Andre Henriques,&nbsp;Alice Simniceanu,&nbsp;Luca Fontana,&nbsp;Andreas Wagner,&nbsp;Stephanie Brazeau,&nbsp;Lisa Waddell","doi":"10.1002/cesm.70042","DOIUrl":"10.1002/cesm.70042","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Public health events of international concern highlight the need for up-to-date evidence curated using sustainable processes that are accessible. In development of the Global Repository of Epidemiological Parameters (grEPI) we explore the performance of an agentic-AI assisted pipeline (GREP-Agent) for screening evidence which capitalizes on recent advancements in large language models (LLMs).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>In this study, the performance of the GREP-Agent was evaluated on a data set of 2000 citations from a systematic review on measles using four LLMs (GPT4o, GPT4o-mini, Llama3.1, and Phi4). The GREP-Agent framework integrates multiple LLMs and human feedback to fine-tune its performance, optimize workload reduction and accuracy in screening research articles. The impact on performance of each part of this Agentic-AI system is presented and measured by accuracy, precision, recall, and F1-score metrics.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>The results show how each phase of the GREP-Agent system incrementally improves accuracy regardless of the LLM. We found that GREP-Agent was able to increase sensitivity across a broad range of open source and proprietary LLMs to 84.2%–88.9% after fine-tuning and to 86.4%–95.3% by varying workload reduction strategies. Performance was significantly impacted by the clarity of the screening questions and setting thresholds for optimized workload reduction strategies.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusions</h3>\u0000 \u0000 <p>The GREP-Agent shows promise in improving the efficiency and effectiveness of evidence synthesis in dynamic public health contexts. Further development and refinement of adaptable human-in-the-loop AI systems for screening literature are essential to support future public health response activities, while maintaining a human-centric approach.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12538819/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145351061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Health Equity in Systematic Reviews: A Tutorial—Part 2 Implementing Health Equity Throughout Your Methods","authors":"Jennifer Petkovic,&nbsp;Jordi Pardo Pardo,&nbsp;Vivian Welch,&nbsp;Omar Dewidar,&nbsp;Lara J. Maxwell,&nbsp;Andrea Darzi,&nbsp;Tamara Lotfi,&nbsp;Lawrence Mbuagbaw,&nbsp;Kevin Pottie,&nbsp;Peter Tugwell","doi":"10.1002/cesm.70054","DOIUrl":"10.1002/cesm.70054","url":null,"abstract":"<p>This is the second and final tutorial in a series on health equity. It provides detailed guidance for considering health equity in systematic reviews of interventions. We will explain how to include and report health equity in all remaining sections of the review.</p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12522172/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145310506","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Long-Term Outcomes of Invasive vs Noninvasive Treatment for Intermittent Claudication: A Systematic Review and Meta-Analysis","authors":"Anas Elmahi,&nbsp;Nathalie Doolan,&nbsp;Mohiedin Hezima,&nbsp;Anwar Gowey,&nbsp;Daragh Moneley,&nbsp;Seamus McHugh,&nbsp;Sayed Aly,&nbsp;Peter Naughton,&nbsp;Elrasheid A. H. Kheirelseid","doi":"10.1002/cesm.70053","DOIUrl":"https://doi.org/10.1002/cesm.70053","url":null,"abstract":"&lt;div&gt;\u0000 \u0000 \u0000 &lt;section&gt;\u0000 \u0000 &lt;h3&gt; Background&lt;/h3&gt;\u0000 \u0000 &lt;p&gt;Intermittent claudication (IC) is a hallmark symptom of peripheral arterial disease (PAD), causing pain and discomfort during physical activity caused by reduced blood flow to the lower extremities. The condition significantly impairs mobility and quality of life (QoL) in affected individuals. Treatment options for IC range from conservative approaches, including best medical therapy (BMT) and supervised exercise therapy (SET), to invasive interventions like angioplasty and open re-vascularization.&lt;/p&gt;\u0000 &lt;/section&gt;\u0000 \u0000 &lt;section&gt;\u0000 \u0000 &lt;h3&gt; Aim&lt;/h3&gt;\u0000 \u0000 &lt;p&gt;This meta-analysis and systematic review seek to assess the long-term results of invasive procedures concerning Noninvasive treatments for the management of patients with IC.&lt;/p&gt;\u0000 &lt;/section&gt;\u0000 \u0000 &lt;section&gt;\u0000 \u0000 &lt;h3&gt; Methods&lt;/h3&gt;\u0000 \u0000 &lt;p&gt;A comprehensive search was conducted in October 2024 across databases containing PubMed, MEDLINE, Cochrane Library, Embase, and Scopus. Randomized controlled trials (RCTs) comparing invasive interventions to Noninvasive treatments were included. Primary outcomes were quality of life (QoL), ankle-brachial pressure index (ABPI), and maximum walking distance (MWD). Secondary outcomes were major adverse cardiovascular events (MACE), mortality, complications, and re-intervention rates. Data analysis was conducted using the Cochrane Review Manager 5. Follow-up duration was between 2 and 7 years, longest available between 2 and 7 years; prioritized 2 years when present.&lt;/p&gt;\u0000 &lt;/section&gt;\u0000 \u0000 &lt;section&gt;\u0000 \u0000 &lt;h3&gt; Results&lt;/h3&gt;\u0000 \u0000 &lt;p&gt;A total of 11 RCTs with 1379 patients were included in the analysis. Invasive treatments demonstrated a significant improvement in MWD and ABPI compared to Noninvasive treatments (MWD pooled Mean Difference (MD) = 64.94 [10.77, 115.12] 95% CI, &lt;i&gt;p&lt;/i&gt; = .02, 5 studies, and ABPI pooled MD = 0.15 [0.04, 0.26] 95% CI, &lt;i&gt;p&lt;/i&gt; = .006, 5 studies). However, invasive interventions were associated with a higher rate of complications, including increased amputation risk (Pooled odds ratio (OR) = 2.46 [0.44, 13.94] 95% CI, &lt;i&gt;p&lt;/i&gt; = .31, 3 studies), though this was not statistically significant. Long-term rates were higher in the Noninvasive treatment group (Pooled OR: 0.56 [0.33, 0.97] 95% CI, &lt;i&gt;p&lt;/i&gt; = .04).&lt;/p&gt;\u0000 &lt;/section&gt;\u0000 \u0000 &lt;section&gt;\u0000 \u0000 &lt;h3&gt; Conclusions&lt;/h3&gt;\u0000 \u0000 &lt;p&gt;Both invasive and Noninvasive treatments are effective in managing IC. Invasive treatments provide greater improvement in blood flow and walking distance, but the risk of complications and re-interventio","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70053","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145224118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comparison of Elicit AI and Traditional Literature Searching in Evidence Syntheses Using Four Case Studies","authors":"Oscar Lau,&nbsp;Su Golder","doi":"10.1002/cesm.70050","DOIUrl":"https://doi.org/10.1002/cesm.70050","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Background</h3>\u0000 \u0000 <p>Elicit AI aims to simplify and accelerate the systematic review process without compromising accuracy. However, research on Elicit's performance is limited.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Objectives</h3>\u0000 \u0000 <p>To determine whether Elicit AI is a viable tool for systematic literature searches and title/abstract screening stages.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>We compared the included studies in four evidence syntheses to those identified using the subscription-based version of Elicit Pro in Review mode. We calculated sensitivity, precision and observed patterns in the performance of Elicit.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>The sensitivity of Elicit was poor, averaging 39.5% (25.5–69.2%) compared to 94.5% (91.1–98.0%) in the original reviews. However, Elicit identified some included studies not identified by the original searches and had an average of 41.8% precision (35.6–46.2%) which was higher than the 7.55% average of the original reviews (0.65–14.7%).</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Discussion</h3>\u0000 \u0000 <p>At the time of this evaluation, Elicit did not search with high enough sensitivity to replace traditional literature searching. However, the high precision of searching in Elicit could prove useful for preliminary searches, and the unique studies identified mean that Elicit can be used by researchers as a useful adjunct.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>Whilst Elicit searches are currently not sensitive enough to replace traditional searching, Elicit is continually improving, and further evaluations should be undertaken as new developments take place.</p>\u0000 </section>\u0000 </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70050","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145172063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Split Body Trials in Systematic Reviews and Meta-Analyses: A Tutorial","authors":"Nuala Livingstone,&nbsp;Kerry Dwan,&nbsp;Marty Chaplin","doi":"10.1002/cesm.70052","DOIUrl":"https://doi.org/10.1002/cesm.70052","url":null,"abstract":"<p>This tutorial focuses on split body trials in the context of a systematic review and meta-analysis. We will explain what split body trials are, the potential unit of analysis issues they can cause, and how to include data from split body trials in a systematic review.</p><p>Split body trials micro learning module\u0000 \u0000 <figure>\u0000 <div><picture>\u0000 <source></source></picture><p></p>\u0000 </div>\u0000 </figure></p>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 5","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70052","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145146377","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}