Can using the Cochrane RCT classifier in EPPI-Reviewer help speed up study selection in qualitative evidence syntheses? A retrospective evaluation

Heather Melanie R. Ames, Christine Hillestad Hestevik, Patricia Sofia Jacobsen Jardim, Martin Smådal Larsen, Lars Jørun Langøien, Hans Bugge Bergsund, Tiril Cecilie Borge
{"title":"Can using the Cochrane RCT classifier in EPPI-Reviewer help speed up study selection in qualitative evidence syntheses? A retrospective evaluation","authors":"Heather Melanie R. Ames,&nbsp;Christine Hillestad Hestevik,&nbsp;Patricia Sofia Jacobsen Jardim,&nbsp;Martin Smådal Larsen,&nbsp;Lars Jørun Langøien,&nbsp;Hans Bugge Bergsund,&nbsp;Tiril Cecilie Borge","doi":"10.1002/cesm.70012","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Introduction</h3>\n \n <p>Using machine learning functions, such as study design classifiers, to automatically identify studies that do not meet the inclusion criteria, is one way to speed up the systematic review screening process. As a qualitative study design classifier is yet to be developed, using the Cochrane randomized controlled trial (RCT) classifier in reverse is one possible way to speed up the identification of primary qualitative studies during screening. The objective of this study was to evaluate whether the Cochrane RCT classifier can be used to speed up the study selection process for qualitative evidence synthesis (QES).</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We performed a retrospective evaluation where we first identified QES. We then extracted the bibliographic information of the included primary qualitative studies in each QES, and uploaded the references into our data management tool, EPPI-Reviewer. We then ran the Cochrane RCT classifier on each group of included studies for each QES.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Eighty-two QES with 2828 unique primary studies were included in the analysis. 56% of the primary studies were classified as unlikely to be an RCT and 40% as being 0–9% likely to be an RCT. 4% were classified as being 10% or more likely to be an RCT. Of these, only 1.7% were classified as being 50% or more likely to be an RCT.</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>The Cochrane RCT classifier could be a useful tool to identify primary studies with qualitative study designs to speed up study selection in a QES. However, it is possible that mixed methods studies or qualitative studies conducted as part of a clinical trial may be missed. Further evaluations using the Cochrane RCT classifier on all the references retrieved from the complete literature search is needed to investigate time- and resource savings.</p>\n </section>\n </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"3 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.70012","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cochrane Evidence Synthesis and Methods","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/cesm.70012","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Introduction

Using machine learning functions, such as study design classifiers, to automatically identify studies that do not meet the inclusion criteria, is one way to speed up the systematic review screening process. As a qualitative study design classifier is yet to be developed, using the Cochrane randomized controlled trial (RCT) classifier in reverse is one possible way to speed up the identification of primary qualitative studies during screening. The objective of this study was to evaluate whether the Cochrane RCT classifier can be used to speed up the study selection process for qualitative evidence synthesis (QES).

Methods

We performed a retrospective evaluation where we first identified QES. We then extracted the bibliographic information of the included primary qualitative studies in each QES, and uploaded the references into our data management tool, EPPI-Reviewer. We then ran the Cochrane RCT classifier on each group of included studies for each QES.

Results

Eighty-two QES with 2828 unique primary studies were included in the analysis. 56% of the primary studies were classified as unlikely to be an RCT and 40% as being 0–9% likely to be an RCT. 4% were classified as being 10% or more likely to be an RCT. Of these, only 1.7% were classified as being 50% or more likely to be an RCT.

Conclusions

The Cochrane RCT classifier could be a useful tool to identify primary studies with qualitative study designs to speed up study selection in a QES. However, it is possible that mixed methods studies or qualitative studies conducted as part of a clinical trial may be missed. Further evaluations using the Cochrane RCT classifier on all the references retrieved from the complete literature search is needed to investigate time- and resource savings.

Abstract Image

求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信