人工智能辅助测试自动化的多年灰色文献综述

IF 4.3 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information and Software Technology Pub Date : 2025-06-06 DOI:10.1016/j.infsof.2025.107799

Filippo Ricca , Alessandro Marchetto , Andrea Stocco

{"title":"人工智能辅助测试自动化的多年灰色文献综述","authors":"Filippo Ricca , Alessandro Marchetto , Andrea Stocco","doi":"10.1016/j.infsof.2025.107799","DOIUrl":null,"url":null,"abstract":"<div><h3>Context:</h3><div>Test Automation (TA) techniques are crucial for quality assurance in software engineering but face limitations such as high test suite maintenance costs and the need for extensive programming skills. Artificial Intelligence (AI) offers new opportunities to address these issues through automation and improved practices.</div></div><div><h3>Objective:</h3><div>Given the prevalent usage of AI in industry, sources of truth are held in grey literature as well as the minds of professionals, stakeholders, developers, and end-users. To this aim, our study surveys grey literature to explore how AI is adopted in TA, focusing on the problems it solves, its solutions, and the available tools. Additionally, the study is complemented by expert insights.</div></div><div><h3>Methods:</h3><div>Over five years, we reviewed over 3,600 grey literature sources, including blogs, white papers, and user manuals, and finally filtered 342 documents to develop taxonomies of TA problems and AI solutions. We also cataloged 100 AI-driven TA tools and interviewed five expert software testers to gain insights into AI’s current and future role in TA.</div></div><div><h3>Results:</h3><div>The study found that manual test code development and maintenance are the main challenges in TA. In contrast, automated test generation and self-healing test scripts are the most common AI solutions. We identified 100 AI-based TA tools, with Applitools, Testim, Functionize, AccelQ, and Mabl being the most adopted in practice.</div></div><div><h3>Conclusion:</h3><div>This paper offers a detailed overview of AI’s impact on TA through grey literature analysis and expert interviews. It presents new taxonomies of TA problems and AI solutions, provides a catalog of AI-driven tools, and relates solutions to problems and tools to solutions. Interview insights further revealed the state and future potential of AI in TA. Our findings support practitioners in selecting TA tools and guide future research.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"186 ","pages":"Article 107799"},"PeriodicalIF":4.3000,"publicationDate":"2025-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A multi-year grey literature review on AI-assisted test automation\",\"authors\":\"Filippo Ricca , Alessandro Marchetto , Andrea Stocco\",\"doi\":\"10.1016/j.infsof.2025.107799\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Context:</h3><div>Test Automation (TA) techniques are crucial for quality assurance in software engineering but face limitations such as high test suite maintenance costs and the need for extensive programming skills. Artificial Intelligence (AI) offers new opportunities to address these issues through automation and improved practices.</div></div><div><h3>Objective:</h3><div>Given the prevalent usage of AI in industry, sources of truth are held in grey literature as well as the minds of professionals, stakeholders, developers, and end-users. To this aim, our study surveys grey literature to explore how AI is adopted in TA, focusing on the problems it solves, its solutions, and the available tools. Additionally, the study is complemented by expert insights.</div></div><div><h3>Methods:</h3><div>Over five years, we reviewed over 3,600 grey literature sources, including blogs, white papers, and user manuals, and finally filtered 342 documents to develop taxonomies of TA problems and AI solutions. We also cataloged 100 AI-driven TA tools and interviewed five expert software testers to gain insights into AI’s current and future role in TA.</div></div><div><h3>Results:</h3><div>The study found that manual test code development and maintenance are the main challenges in TA. In contrast, automated test generation and self-healing test scripts are the most common AI solutions. We identified 100 AI-based TA tools, with Applitools, Testim, Functionize, AccelQ, and Mabl being the most adopted in practice.</div></div><div><h3>Conclusion:</h3><div>This paper offers a detailed overview of AI’s impact on TA through grey literature analysis and expert interviews. It presents new taxonomies of TA problems and AI solutions, provides a catalog of AI-driven tools, and relates solutions to problems and tools to solutions. Interview insights further revealed the state and future potential of AI in TA. Our findings support practitioners in selecting TA tools and guide future research.</div></div>\",\"PeriodicalId\":54983,\"journal\":{\"name\":\"Information and Software Technology\",\"volume\":\"186 \",\"pages\":\"Article 107799\"},\"PeriodicalIF\":4.3000,\"publicationDate\":\"2025-06-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Information and Software Technology\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0950584925001387\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information and Software Technology","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0950584925001387","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

摘要

背景：测试自动化（TA）技术对于软件工程中的质量保证是至关重要的，但是面临着诸如高测试套件维护成本和对广泛编程技能的需求等限制。人工智能（AI）通过自动化和改进的实践为解决这些问题提供了新的机会。目标：考虑到人工智能在工业中的普遍使用，真相的来源存在于灰色文献中，以及专业人士、利益相关者、开发人员和最终用户的头脑中。为此，我们的研究调查了灰色文献，以探索AI如何在TA中被采用，重点关注它解决的问题，它的解决方案和可用的工具。此外，该研究还得到了专家见解的补充。方法：在5年多的时间里，我们回顾了3600多篇灰色文献，包括博客、白皮书和用户手册，最终过滤了342篇文档，建立了TA问题和AI解决方案的分类。我们还对100个AI驱动的TA工具进行了分类，并采访了5位软件测试专家，以深入了解AI在TA中当前和未来的角色。结果：研究发现手工测试代码的开发和维护是测试技术的主要挑战。相比之下，自动化测试生成和自修复测试脚本是最常见的AI解决方案。我们确定了100个基于ai的TA工具，其中applittools、testm、Functionize、AccelQ和Mabl在实践中被采用得最多。结论：本文通过灰色文献分析和专家访谈，详细概述了AI对TA的影响。它提出了人工智能问题和人工智能解决方案的新分类，提供了人工智能驱动工具的目录，并将问题的解决方案和解决方案的工具联系起来。访谈洞察进一步揭示了AI在TA领域的现状和未来潜力。我们的发现支持从业者选择TA工具，并指导未来的研究。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

A multi-year grey literature review on AI-assisted test automation

Context:

Test Automation (TA) techniques are crucial for quality assurance in software engineering but face limitations such as high test suite maintenance costs and the need for extensive programming skills. Artificial Intelligence (AI) offers new opportunities to address these issues through automation and improved practices.

Objective:

Given the prevalent usage of AI in industry, sources of truth are held in grey literature as well as the minds of professionals, stakeholders, developers, and end-users. To this aim, our study surveys grey literature to explore how AI is adopted in TA, focusing on the problems it solves, its solutions, and the available tools. Additionally, the study is complemented by expert insights.

Methods:

Over five years, we reviewed over 3,600 grey literature sources, including blogs, white papers, and user manuals, and finally filtered 342 documents to develop taxonomies of TA problems and AI solutions. We also cataloged 100 AI-driven TA tools and interviewed five expert software testers to gain insights into AI’s current and future role in TA.

Results:

The study found that manual test code development and maintenance are the main challenges in TA. In contrast, automated test generation and self-healing test scripts are the most common AI solutions. We identified 100 AI-based TA tools, with Applitools, Testim, Functionize, AccelQ, and Mabl being the most adopted in practice.

Conclusion:

This paper offers a detailed overview of AI’s impact on TA through grey literature analysis and expert interviews. It presents new taxonomies of TA problems and AI solutions, provides a catalog of AI-driven tools, and relates solutions to problems and tools to solutions. Interview insights further revealed the state and future potential of AI in TA. Our findings support practitioners in selecting TA tools and guide future research.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Information and Software Technology 工程技术-计算机：软件工程

CiteScore

9.10

自引率

7.70%

发文量

164

审稿时长

9.6 weeks

期刊介绍： Information and Software Technology is the international archival journal focusing on research and experience that contributes to the improvement of software development practices. The journal''s scope includes methods and techniques to better engineer software and manage its development. Articles submitted for review should have a clear component of software engineering or address ways to improve the engineering and management of software development. Areas covered by the journal include: • Software management, quality and metrics, • Software processes, • Software architecture, modelling, specification, design and programming • Functional and non-functional software requirements • Software testing and verification & validation • Empirical studies of all aspects of engineering and managing software development Short Communications is a new section dedicated to short papers addressing new ideas, controversial opinions, "Negative" results and much more. Read the Guide for authors for more information. The journal encourages and welcomes submissions of systematic literature studies (reviews and maps) within the scope of the journal. Information and Software Technology is the premiere outlet for systematic literature studies in software engineering.