ISTA+: Test case generation and optimization for intelligent systems based on coverage analysis

IF 1.5 4区计算机科学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Science of Computer Programming Pub Date : 2024-01-12 DOI:10.1016/j.scico.2024.103078

Xiaoxue Wu , Yizeng Gu , Lidan Lin , Wei Zheng , Xiang Chen

{"title":"ISTA+: Test case generation and optimization for intelligent systems based on coverage analysis","authors":"Xiaoxue Wu , Yizeng Gu , Lidan Lin , Wei Zheng , Xiang Chen","doi":"10.1016/j.scico.2024.103078","DOIUrl":null,"url":null,"abstract":"<div><p>With the increasing use of intelligent systems in various domains such as self-driving cars, robotics, and smart cities, it is crucial to ensure the quality of intelligent systems for their reliable and effective use in various domains. However, testing intelligent systems poses unique challenges due to their complex structure, low efficiency, and the high cost associated with manually collecting a large number of test cases. Hence, it is crucial to design tools that can adequately test intelligent systems while overcoming these obstacles.</p><p>We propose an intelligent system test tool called ISTA+. This tool implements automatic generation and optimization of test cases based on coverage analysis, resulting in improved test adequacy for intelligent systems. To evaluate the effectiveness of ISTA+, we applied it to two different models (fully-connected DNN and the Rambo model) and two datasets of different data types (i.e., image and text). The evaluation results demonstrate that ISTA+ successfully improves the test dataset quality and ensures comprehensive testing for both text and image data types.</p><ul><li><span>•</span><span><p>Link to source code: <span>https://github.com/wuxiaoxue/ISTAplus</span><svg><path></path></svg></p></span></li><li><span>•</span><span><p>Link to video demonstration: <span>https://youtu.be/6CkzMJ0ghq8</span><svg><path></path></svg></p></span></li></ul></div>","PeriodicalId":49561,"journal":{"name":"Science of Computer Programming","volume":"234 ","pages":"Article 103078"},"PeriodicalIF":1.5000,"publicationDate":"2024-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science of Computer Programming","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167642324000017","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

With the increasing use of intelligent systems in various domains such as self-driving cars, robotics, and smart cities, it is crucial to ensure the quality of intelligent systems for their reliable and effective use in various domains. However, testing intelligent systems poses unique challenges due to their complex structure, low efficiency, and the high cost associated with manually collecting a large number of test cases. Hence, it is crucial to design tools that can adequately test intelligent systems while overcoming these obstacles.

We propose an intelligent system test tool called ISTA+. This tool implements automatic generation and optimization of test cases based on coverage analysis, resulting in improved test adequacy for intelligent systems. To evaluate the effectiveness of ISTA+, we applied it to two different models (fully-connected DNN and the Rambo model) and two datasets of different data types (i.e., image and text). The evaluation results demonstrate that ISTA+ successfully improves the test dataset quality and ensures comprehensive testing for both text and image data types.

•
Link to source code: https://github.com/wuxiaoxue/ISTAplus
•
Link to video demonstration: https://youtu.be/6CkzMJ0ghq8

查看原文本刊更多论文

ISTA+：基于覆盖率分析的智能系统测试用例生成与优化

随着智能系统在自动驾驶汽车、机器人和智能城市等各个领域的应用日益广泛，确保智能系统的质量，使其在各个领域得到可靠、有效的应用至关重要。然而，由于智能系统结构复杂、效率低下，而且人工收集大量测试用例的成本较高，因此测试智能系统面临着独特的挑战。因此，设计既能充分测试智能系统又能克服这些障碍的工具至关重要。我们提出了一种名为 ISTA+ 的智能系统测试工具。该工具基于覆盖率分析实现了测试用例的自动生成和优化，从而提高了智能系统测试的充分性。为了评估 ISTA+ 的有效性，我们将其应用于两种不同的模型（全连接 DNN 和 Rambo 模型）和两种不同数据类型的数据集（即图像和文本）。评估结果表明，ISTA+ 成功地提高了测试数据集的质量，并确保了对文本和图像数据类型的全面测试。-源代码链接：https://github.com/wuxiaoxue/ISTAplus-Link 视频演示：https://youtu.be/6CkzMJ0ghq8

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Science of Computer Programming 工程技术-计算机：软件工程

CiteScore

3.80

自引率

0.00%

发文量

审稿时长

67 days

期刊介绍： Science of Computer Programming is dedicated to the distribution of research results in the areas of software systems development, use and maintenance, including the software aspects of hardware design. The journal has a wide scope ranging from the many facets of methodological foundations to the details of technical issues andthe aspects of industrial practice. The subjects of interest to SCP cover the entire spectrum of methods for the entire life cycle of software systems, including • Requirements, specification, design, validation, verification, coding, testing, maintenance, metrics and renovation of software; • Design, implementation and evaluation of programming languages; • Programming environments, development tools, visualisation and animation; • Management of the development process; • Human factors in software, software for social interaction, software for social computing; • Cyber physical systems, and software for the interaction between the physical and the machine; • Software aspects of infrastructure services, system administration, and network management.