From manual to automated systematic review: Key attributes influencing the duration of systematic reviews in software engineering

IF 3.1 2区计算机科学 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Computer Standards & Interfaces Pub Date : 2025-09-12 DOI:10.1016/j.csi.2025.104073

Diana Kalibatiene , Jolanta Miliauskaitė

{"title":"From manual to automated systematic review: Key attributes influencing the duration of systematic reviews in software engineering","authors":"Diana Kalibatiene , Jolanta Miliauskaitė","doi":"10.1016/j.csi.2025.104073","DOIUrl":null,"url":null,"abstract":"<div><h3>Context</h3><div>It is widely accepted that a systematic literature review (SLR) is an effective, comprehensive, repeatable, less biased, and transparent method for gathering and condensing knowledge from existing scientific publications. The SLR method facilitates the identification of gaps for new research opportunities, fostering decision-making based on evidence. At the same time, SLR is a time and effort-consuming task that is threatened by the increasing volume of publications. However, there is a lack of comprehension about which factors directly impact manually conducted SLR performance, limiting researchers to better plan and optimize their processes.</div></div><div><h3>Objective</h3><div>To enhance the understanding of the attributes that directly influence the SLR process in terms of time consumption.</div></div><div><h3>Methods</h3><div>We performed a tertiary study that (i) identified 138 secondary studies, (ii) mapped the possible influential attributes for SLR performance, (iii) extracted data from SLR reports and metadata, synthesized and analysed their influence, providing an overview of core trends related to those attributes over time.</div></div><div><h3>Results</h3><div>Our SLR mapped four main attributes influencing the performance time of the SLR process – number of authors, number of initially retrieved papers from databases, number of included papers for data synthesis, and usage of the snowballing techniques. We noticed a trend for smaller research groups (2–5 persons) using 4–6 different databases and processing to process a large number of studies, and an increasing adoption of the snowballing technique.</div></div><div><h3>Conclusion</h3><div>This paper reveals a bottleneck in manually conducted SLR, reinforcing the need for evolving automation. Mapping the attributes is only the first step to making the SLR process more measurable regarding its resource consumption. We contribute by providing recommendations to assist scientists and practitioners in planning their future SLRs and IT projects, including SLRs, particularly in the evolving landscape of digital transformation and innovations.</div></div>","PeriodicalId":50635,"journal":{"name":"Computer Standards & Interfaces","volume":"96 ","pages":"Article 104073"},"PeriodicalIF":3.1000,"publicationDate":"2025-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Standards & Interfaces","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0920548925001023","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}

引用次数: 0

Abstract

Context

It is widely accepted that a systematic literature review (SLR) is an effective, comprehensive, repeatable, less biased, and transparent method for gathering and condensing knowledge from existing scientific publications. The SLR method facilitates the identification of gaps for new research opportunities, fostering decision-making based on evidence. At the same time, SLR is a time and effort-consuming task that is threatened by the increasing volume of publications. However, there is a lack of comprehension about which factors directly impact manually conducted SLR performance, limiting researchers to better plan and optimize their processes.

Objective

To enhance the understanding of the attributes that directly influence the SLR process in terms of time consumption.

Methods

We performed a tertiary study that (i) identified 138 secondary studies, (ii) mapped the possible influential attributes for SLR performance, (iii) extracted data from SLR reports and metadata, synthesized and analysed their influence, providing an overview of core trends related to those attributes over time.

Results

Our SLR mapped four main attributes influencing the performance time of the SLR process – number of authors, number of initially retrieved papers from databases, number of included papers for data synthesis, and usage of the snowballing techniques. We noticed a trend for smaller research groups (2–5 persons) using 4–6 different databases and processing to process a large number of studies, and an increasing adoption of the snowballing technique.

Conclusion

This paper reveals a bottleneck in manually conducted SLR, reinforcing the need for evolving automation. Mapping the attributes is only the first step to making the SLR process more measurable regarding its resource consumption. We contribute by providing recommendations to assist scientists and practitioners in planning their future SLRs and IT projects, including SLRs, particularly in the evolving landscape of digital transformation and innovations.

查看原文本刊更多论文

从手动到自动的系统评审：影响软件工程中系统评审持续时间的关键属性

人们普遍认为，系统文献综述（SLR）是一种有效、全面、可重复、较少偏见和透明的方法，可以从现有的科学出版物中收集和浓缩知识。SLR方法有助于识别新的研究机会的差距，促进基于证据的决策。与此同时，单反是一项耗时费力的任务，受到越来越多的出版物的威胁。然而，对于哪些因素会直接影响手动单反相机的性能，人们缺乏理解，这限制了研究人员更好地规划和优化他们的过程。目的提高对直接影响单反过程的时间消耗属性的认识。方法我们进行了一项三级研究，其中(i)确定了138项二级研究，（ii）绘制了可能影响SLR性能的属性图，（iii）从SLR报告和元数据中提取数据，综合并分析了它们的影响，概述了与这些属性相关的核心趋势。结果我们的SLR映射了影响SLR过程性能时间的四个主要属性：作者数量、数据库初始检索论文数量、数据综合纳入论文数量和滚雪球技术的使用。我们注意到一个趋势，小型研究小组（2-5人）使用4-6个不同的数据库和处理方法来处理大量的研究，并且越来越多地采用滚雪球技术。本文揭示了人工单反的瓶颈，加强了自动化发展的必要性。映射属性只是使SLR过程在其资源消耗方面更加可测量的第一步。我们的贡献是为科学家和从业者提供建议，帮助他们规划未来的单反和IT项目，包括单反，特别是在不断发展的数字化转型和创新环境中。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Computer Standards & Interfaces 工程技术-计算机：软件工程

CiteScore

11.90

自引率

16.00%

发文量

审稿时长

6 months

期刊介绍： The quality of software, well-defined interfaces (hardware and software), the process of digitalisation, and accepted standards in these fields are essential for building and exploiting complex computing, communication, multimedia and measuring systems. Standards can simplify the design and construction of individual hardware and software components and help to ensure satisfactory interworking. Computer Standards & Interfaces is an international journal dealing specifically with these topics. The journal • Provides information about activities and progress on the definition of computer standards, software quality, interfaces and methods, at national, European and international levels • Publishes critical comments on standards and standards activities • Disseminates user''s experiences and case studies in the application and exploitation of established or emerging standards, interfaces and methods • Offers a forum for discussion on actual projects, standards, interfaces and methods by recognised experts • Stimulates relevant research by providing a specialised refereed medium.