Carolline Pena, Bruno Cartaxo, Igor Steinmacher, Deepika Badampudi, Deyvson da Silva, Williby Ferreira, Adauto Almeida, Fernando Kamei, Sérgio Soares
{"title":"Comparing the Efficacy of Rapid Review With a Systematic Review in the Software Engineering Field","authors":"Carolline Pena, Bruno Cartaxo, Igor Steinmacher, Deepika Badampudi, Deyvson da Silva, Williby Ferreira, Adauto Almeida, Fernando Kamei, Sérgio Soares","doi":"10.1002/smr.2748","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Context</h3>\n \n <p>Rapid Reviews are secondary studies aiming to deliver evidence to experts in a more timely manner and with lower costs than traditional literature reviews. Previous studies have shown that experts and researchers are positive toward Rapid Reviews. However, little is known about how Rapid Reviews differ from traditional Systematic Reviews.</p>\n </section>\n \n <section>\n \n <h3> Objective</h3>\n \n <p>The goal of this paper is to compare a Rapid Review with a Systematic Review in terms of their methods (e.g., search strategy, study selection, quality assessment, and data extraction) and findings to understand how optimizing the traditional Systematic Review method impacts what we obtain with Rapid Review.</p>\n </section>\n \n <section>\n \n <h3> Method</h3>\n \n <p>To achieve this goal, we conducted a Systematic Review with the same research questions answered by a pre-existing Rapid Review and compared those two studies. Also, we surveyed experts from industry and academia to evaluate the relevance of the findings obtained from both the secondary studies.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>The Rapid Review lasted 6 days, while the Systematic Review took 1 year and 2 months. The main bottlenecks we identified in the Systematic Review are (i) executing the search strategy and (ii) selecting the procedure. Together, they took 10 months. The researchers had to analyze the information from 11,383 papers for the Systematic Review compared with 1973 for the Rapid Review. Still, most (<span></span><math>\n <semantics>\n <mrow>\n <mo>∼</mo>\n </mrow>\n <annotation>$$ \\sim $$</annotation>\n </semantics></math> 78%) of the papers included in the Systematic Review were returned by the Rapid Review search, and some papers that could be included were unduly excluded during the Rapid Review's selection procedure. Both secondary studies identified the same number of pieces of evidence (30), but the pieces of evidence are not the same.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>The Rapid Review and Systematic Review results are inherently different and complementary. The time and cost to conduct a Systematic Review can be prohibitive in experts' contexts. Thus, at least in such situations, a Rapid Review may be an adequate choice. Moreover, a Rapid Review may be executed in the experts' context as a previous low-cost step before deciding to invest in a high-cost Systematic Review.</p>\n </section>\n </div>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"37 1","pages":""},"PeriodicalIF":1.7000,"publicationDate":"2024-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Software-Evolution and Process","FirstCategoryId":"94","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/smr.2748","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0
Abstract
Context
Rapid Reviews are secondary studies aiming to deliver evidence to experts in a more timely manner and with lower costs than traditional literature reviews. Previous studies have shown that experts and researchers are positive toward Rapid Reviews. However, little is known about how Rapid Reviews differ from traditional Systematic Reviews.
Objective
The goal of this paper is to compare a Rapid Review with a Systematic Review in terms of their methods (e.g., search strategy, study selection, quality assessment, and data extraction) and findings to understand how optimizing the traditional Systematic Review method impacts what we obtain with Rapid Review.
Method
To achieve this goal, we conducted a Systematic Review with the same research questions answered by a pre-existing Rapid Review and compared those two studies. Also, we surveyed experts from industry and academia to evaluate the relevance of the findings obtained from both the secondary studies.
Results
The Rapid Review lasted 6 days, while the Systematic Review took 1 year and 2 months. The main bottlenecks we identified in the Systematic Review are (i) executing the search strategy and (ii) selecting the procedure. Together, they took 10 months. The researchers had to analyze the information from 11,383 papers for the Systematic Review compared with 1973 for the Rapid Review. Still, most ( 78%) of the papers included in the Systematic Review were returned by the Rapid Review search, and some papers that could be included were unduly excluded during the Rapid Review's selection procedure. Both secondary studies identified the same number of pieces of evidence (30), but the pieces of evidence are not the same.
Conclusion
The Rapid Review and Systematic Review results are inherently different and complementary. The time and cost to conduct a Systematic Review can be prohibitive in experts' contexts. Thus, at least in such situations, a Rapid Review may be an adequate choice. Moreover, a Rapid Review may be executed in the experts' context as a previous low-cost step before deciding to invest in a high-cost Systematic Review.
快速综述是一种二级研究，旨在比传统文献综述更及时、成本更低地向专家提供证据。先前的研究表明，专家和研究人员对快速评论持积极态度。然而，很少有人知道快速审查与传统的系统审查有什么不同。本文的目的是比较快速综述和系统综述的方法（如搜索策略、研究选择、质量评估和数据提取）和结果，以了解优化传统的系统综述方法如何影响我们通过快速综述获得的结果。方法为了实现这一目标，我们对已有的快速综述回答的相同研究问题进行了系统综述，并对这两项研究进行了比较。此外，我们还调查了来自工业界和学术界的专家，以评估从这两项二次研究中获得的结果的相关性。结果快速评价耗时6天，系统评价耗时1年零2个月。我们在系统综述中发现的主要瓶颈是(i)执行搜索策略和（ii）选择程序。他们一起花了10个月。研究人员分析了11383篇论文的系统评价信息，与1973年的快速评价进行了比较。但是，大部分（～ $$ \sim $$ 78）%) of the papers included in the Systematic Review were returned by the Rapid Review search, and some papers that could be included were unduly excluded during the Rapid Review's selection procedure. Both secondary studies identified the same number of pieces of evidence (30), but the pieces of evidence are not the same. Conclusion The Rapid Review and Systematic Review results are inherently different and complementary. The time and cost to conduct a Systematic Review can be prohibitive in experts' contexts. Thus, at least in such situations, a Rapid Review may be an adequate choice. Moreover, a Rapid Review may be executed in the experts' context as a previous low-cost step before deciding to invest in a high-cost Systematic Review.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
