Charlie Holland, Daniel B Oakes, Mohinder Sarna, Kevin Ek Chai, Leo Ng, Hannah C Moore
{"title":"Validity of using a semi-automated screening tool in a systematic review assessing non-specific effects of respiratory vaccines.","authors":"Charlie Holland, Daniel B Oakes, Mohinder Sarna, Kevin Ek Chai, Leo Ng, Hannah C Moore","doi":"10.1186/s12874-025-02511-7","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The abstract screening process of systematic reviews can take thousands of hours by two researchers. We aim to determine the reliability and validity of Research Screener, a semi-automated abstract screening tool within a systematic review on non-specific and broader effects of respiratory vaccines on acute lower respiratory infection hospitalisations and antimicrobial prescribing patterns in young children.</p><p><strong>Methods: </strong>We searched online databases for Medline, Embase, CINAHL, Scopus and ClinicalTrials.gov from inception until 24th January 2024. We included human studies involving non-specific and broader effects of respiratory vaccines and excluded studies investigating live-attenuated vaccines. The RS trial compared relevant abstracts flagged by RS to manual screening. RS ranks abstracts by relevance based on seed articles used to validate the search strategy. Abstracts are re-ranked following reviewers' feedback. Two reviewers screened RS independently with a third reviewer resolving conflicts; three reviewers screened manually with a fourth reviewer resolving conflicts.</p><p><strong>Results: </strong>After removal of duplicates, 9,727 articles were identified for abstract screening. Of those, 3,000 were randomly selected for screening in RS, with 18% (540) screened in RS and 100% manually. In RS, 99 relevant articles were identified. After comparing RS to manual screening and completing full-text review on 26 articles not captured by RS, 4 articles were missed by RS (2 due to human error, 2 not yet screened). Hence, RS captured articles accurately whilst reducing the screening load.</p><p><strong>Conclusions: </strong>RS is a valid and reliable tool that reduces the amount of time spent screening articles for large-scale systematic reviews. RS is a useful tool that should be considered for streamlining the process of systematic reviews.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"59"},"PeriodicalIF":3.9000,"publicationDate":"2025-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11884005/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Medical Research Methodology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12874-025-02511-7","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0
Abstract
Background: The abstract screening process of systematic reviews can take thousands of hours by two researchers. We aim to determine the reliability and validity of Research Screener, a semi-automated abstract screening tool within a systematic review on non-specific and broader effects of respiratory vaccines on acute lower respiratory infection hospitalisations and antimicrobial prescribing patterns in young children.
Methods: We searched online databases for Medline, Embase, CINAHL, Scopus and ClinicalTrials.gov from inception until 24th January 2024. We included human studies involving non-specific and broader effects of respiratory vaccines and excluded studies investigating live-attenuated vaccines. The RS trial compared relevant abstracts flagged by RS to manual screening. RS ranks abstracts by relevance based on seed articles used to validate the search strategy. Abstracts are re-ranked following reviewers' feedback. Two reviewers screened RS independently with a third reviewer resolving conflicts; three reviewers screened manually with a fourth reviewer resolving conflicts.
Results: After removal of duplicates, 9,727 articles were identified for abstract screening. Of those, 3,000 were randomly selected for screening in RS, with 18% (540) screened in RS and 100% manually. In RS, 99 relevant articles were identified. After comparing RS to manual screening and completing full-text review on 26 articles not captured by RS, 4 articles were missed by RS (2 due to human error, 2 not yet screened). Hence, RS captured articles accurately whilst reducing the screening load.
Conclusions: RS is a valid and reliable tool that reduces the amount of time spent screening articles for large-scale systematic reviews. RS is a useful tool that should be considered for streamlining the process of systematic reviews.
期刊介绍：
BMC Medical Research Methodology is an open access journal publishing original peer-reviewed research articles in methodological approaches to healthcare research. Articles on the methodology of epidemiological research, clinical trials and meta-analysis/systematic review are particularly encouraged, as are empirical studies of the associations between choice of methodology and study outcomes. BMC Medical Research Methodology does not aim to publish articles describing scientific methods or techniques: these should be directed to the BMC journal covering the relevant biomedical subject area.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
