Predicting test failures induced by software defects: A lightweight alternative to software defect prediction and its industrial application

IF 3.7 2区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Journal of Systems and Software Pub Date : 2025-02-03 DOI:10.1016/j.jss.2025.112360

Lech Madeyski , Szymon Stradowski

{"title":"Predicting test failures induced by software defects: A lightweight alternative to software defect prediction and its industrial application","authors":"Lech Madeyski , Szymon Stradowski","doi":"10.1016/j.jss.2025.112360","DOIUrl":null,"url":null,"abstract":"<div><h3>Context:</h3><div>Machine Learning Software Defect Prediction (ML SDP) is a promising method to improve the quality and minimise the cost of software development.</div></div><div><h3>Objective:</h3><div>We aim to: (1) apropose and develop a Lightweight Alternative to SDP (LA2SDP) that predicts test failures induced by software defects to allow pinpointing defective software modules thanks to available mapping of predicted test failures to past defects and corrected modules, (2) preliminary evaluate the proposed method in a real-world Nokia 5G scenario.</div></div><div><h3>Method:</h3><div>We train machine learning models using test failures that come from confirmed software defects already available in the Nokia 5G environment. We implement LA2SDP using five supervised ML algorithms, together with their tuned versions, and use eXplainable AI (XAI) to provide feedback to stakeholders and initiate quality improvement actions.</div></div><div><h3>Results:</h3><div>We have shown that LA2SDP is feasible in vivo using test failure-to-defect report mapping readily available within the Nokia 5G system-level test process, achieving good predictive performance. Specifically, CatBoost Gradient Boosting turned out to perform the best and achieved satisfactory Matthew’s Correlation Coefficient (MCC) results for our feasibility study.</div></div><div><h3>Conclusions:</h3><div>Our efforts have successfully defined, developed, and validated LA2SDP, using the sliding and expanding window approaches on an industrial data set.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"223 ","pages":"Article 112360"},"PeriodicalIF":3.7000,"publicationDate":"2025-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Systems and Software","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0164121225000287","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

Context:

Machine Learning Software Defect Prediction (ML SDP) is a promising method to improve the quality and minimise the cost of software development.

Objective:

We aim to: (1) apropose and develop a Lightweight Alternative to SDP (LA2SDP) that predicts test failures induced by software defects to allow pinpointing defective software modules thanks to available mapping of predicted test failures to past defects and corrected modules, (2) preliminary evaluate the proposed method in a real-world Nokia 5G scenario.

Method:

We train machine learning models using test failures that come from confirmed software defects already available in the Nokia 5G environment. We implement LA2SDP using five supervised ML algorithms, together with their tuned versions, and use eXplainable AI (XAI) to provide feedback to stakeholders and initiate quality improvement actions.

Results:

We have shown that LA2SDP is feasible in vivo using test failure-to-defect report mapping readily available within the Nokia 5G system-level test process, achieving good predictive performance. Specifically, CatBoost Gradient Boosting turned out to perform the best and achieved satisfactory Matthew’s Correlation Coefficient (MCC) results for our feasibility study.

Conclusions:

Our efforts have successfully defined, developed, and validated LA2SDP, using the sliding and expanding window approaches on an industrial data set.

Abstract Image

查看原文本刊更多论文

预测由软件缺陷引起的测试失败：软件缺陷预测及其工业应用的轻量级替代方案

背景：机器学习软件缺陷预测（ML SDP）是一种很有前途的方法，可以提高软件开发的质量并最大限度地降低成本。目的：我们的目标是：(1)提出并开发一种轻量级的SDP替代方案（LA2SDP），该方案可以预测由软件缺陷引起的测试故障，从而通过预测测试故障到过去缺陷和纠正模块的可用映射来精确定位有缺陷的软件模块；(2)在真实的诺基亚5G场景中初步评估所提出的方法。方法：我们使用测试失败来训练机器学习模型，这些失败来自诺基亚5G环境中已经存在的已确认的软件缺陷。我们使用五种有监督的ML算法及其调整版本来实现LA2SDP，并使用可解释的AI （XAI）向利益相关者提供反馈并启动质量改进行动。结果：我们已经证明LA2SDP在体内是可行的，使用诺基亚5G系统级测试过程中现成的测试故障到缺陷报告映射，取得了良好的预测性能。具体来说，CatBoost梯度增强在我们的可行性研究中表现最好，并获得了令人满意的马修相关系数（MCC）结果。结论：我们的努力已经成功地定义、开发和验证了LA2SDP，在工业数据集上使用滑动和扩展窗口方法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Systems and Software 工程技术-计算机：理论方法

CiteScore

8.60

自引率

5.70%

发文量

193

审稿时长

16 weeks

期刊介绍： The Journal of Systems and Software publishes papers covering all aspects of software engineering and related hardware-software-systems issues. All articles should include a validation of the idea presented, e.g. through case studies, experiments, or systematic comparisons with other approaches already in practice. Topics of interest include, but are not limited to: •Methods and tools for, and empirical studies on, software requirements, design, architecture, verification and validation, maintenance and evolution •Agile, model-driven, service-oriented, open source and global software development •Approaches for mobile, multiprocessing, real-time, distributed, cloud-based, dependable and virtualized systems •Human factors and management concerns of software development •Data management and big data issues of software systems •Metrics and evaluation, data mining of software development resources •Business and economic aspects of software development processes The journal welcomes state-of-the-art surveys and reports of practical experience for all of these topics.