Information and Software Technology最新文献

筛选
英文 中文
Markov model based coverage testing of deep learning software systems 基于马尔可夫模型的深度学习软件系统覆盖率测试
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-23 DOI: 10.1016/j.infsof.2024.107628
Ying Shi, Beibei Yin, Jing-Ao Shi
{"title":"Markov model based coverage testing of deep learning software systems","authors":"Ying Shi,&nbsp;Beibei Yin,&nbsp;Jing-Ao Shi","doi":"10.1016/j.infsof.2024.107628","DOIUrl":"10.1016/j.infsof.2024.107628","url":null,"abstract":"<div><h3>Context:</h3><div>Deep Learning (DL) software systems have been widely deployed in safety and security-critical domains, which calls for systematic testing to guarantee their accuracy and reliability. Objective measurement of test quality is one of the key issues in software testing. Recently, many coverage criteria have been proposed to measure the testing adequacy of Deep Neural Networks (DNNs).</div></div><div><h3>Objective:</h3><div>Recent research demonstrates that existing criteria have some limitations on interpreting the increasingly diverse behaviors of DNNs or clarifying the relationship between the coverage and the decision logic of DNNs. Moreover, some evaluations argue against the correlation between coverage and defect detection. In this paper, a novel coverage approach is proposed to interpret the internal information of programs.</div></div><div><h3>Methods:</h3><div>The process of coverage testing is formalized and quantified by constructing Markov models based on critical neurons extracted using Layer-wise Relevance Propagation in the structure of DNNs. The difference in the transition matrix of Markov chains between training and testing data is measured by KL divergence, and it is developed as a coverage criterion.</div></div><div><h3>Results:</h3><div>The values of the proposed coverage increase as the number of classes increases. The values are different for various test suites, and they become higher with the addition of new samples. Higher coverage values are observed to correlate with an increased fault detection capability.</div></div><div><h3>Conclusion:</h3><div>The experimental results illustrate that the proposed approach can effectively measure actual diversity and exhibit more adaptability to additional test cases. Furthermore, there is a positive correlation between the proposed coverage and fault detection, which provides support for test case selection guided by coverage.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107628"},"PeriodicalIF":3.8,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142722027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An alternative to code comment generation? Generating comment from bytecode 代码注释生成的替代方案?从字节码生成注释
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-22 DOI: 10.1016/j.infsof.2024.107623
Xiangping Chen , Junqi Chen , Zhilu Lian , Yuan Huang , Xiaocong Zhou , Yunzhi Wu , Zibin Zheng
{"title":"An alternative to code comment generation? Generating comment from bytecode","authors":"Xiangping Chen ,&nbsp;Junqi Chen ,&nbsp;Zhilu Lian ,&nbsp;Yuan Huang ,&nbsp;Xiaocong Zhou ,&nbsp;Yunzhi Wu ,&nbsp;Zibin Zheng","doi":"10.1016/j.infsof.2024.107623","DOIUrl":"10.1016/j.infsof.2024.107623","url":null,"abstract":"<div><h3>Context:</h3><div>Due to the importance and necessity of code comments, recent works propose many comment generation models with source code as input. But sometimes there has no access to obtain the source code, only the bytecode, such as many Apps.</div></div><div><h3>Objective:</h3><div>If there is a way to generate comments for bytecode directly, tasks such as malware detection and understanding closed-source software can benefit from the generated comment because it improves the understandability of the system. Therefore, we propose a novel approach called ByteGen to generate comments from bytecode.</div></div><div><h3>Methods:</h3><div>Specifically, to extract the structure characteristic of the bytecode, we utilize the control flow graph (CFG) of the bytecode and use a special traversal named enhanced SBT to serialize CFG. The enhanced SBT can completely preserve the structure of the CFG in a sequence. We set up experiments on a dataset with a scale of about 50,000 bytecode-comment pairs collected from Maven.</div></div><div><h3>Results:</h3><div>Experimental results show that the average BLEU-4 score of ByteGen is 28.67, which outperforms several baselines, and a human study also indicates the effectiveness of ByteGen in generating comments from bytecodes.</div></div><div><h3>Conclusion:</h3><div>In general, ByteGen performs better than other baselines. Therefore, this also proves the effectiveness of our approach in the code comment generation scenario without source code.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107623"},"PeriodicalIF":3.8,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142748030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Improving operational decision-making through decision mining - utilizing method engineering for the creation of a decision mining method 通过决策挖掘改进业务决策--利用方法工程创建决策挖掘方法
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-19 DOI: 10.1016/j.infsof.2024.107627
Sam Leewis , Koen Smit , Bas van den Boom , Johan Versendaal
{"title":"Improving operational decision-making through decision mining - utilizing method engineering for the creation of a decision mining method","authors":"Sam Leewis ,&nbsp;Koen Smit ,&nbsp;Bas van den Boom ,&nbsp;Johan Versendaal","doi":"10.1016/j.infsof.2024.107627","DOIUrl":"10.1016/j.infsof.2024.107627","url":null,"abstract":"<div><h3>Context</h3><div>This study addresses the challenge of enhancing the efficiency and agility of decision support software supporting both operational decision-making and software production teams developing decision support software. It centers on creating a method that assists in mining decisions, checking decisions on conformance, and improving decisions, which supports software production teams in developing decision support software.</div></div><div><h3>Objective</h3><div>The primary objective is to develop an explicit, clear, and structured approach for discovering, checking, and improving decisions using decision support software. The study aims to create a blueprint for software production teams to develop Decision Mining (DM) software, in line with recent advancements in the field. Additionally, it seeks to provide a consolidated, methodical overview of activities and deliverables in the DM research field.</div></div><div><h3>Method</h3><div>The research employs method engineering principles to construct a method for DM that leverages the existing body of knowledge by utilizing a Systematic Literature Review (SLR). The study focuses on developing individual building blocks and method fragments incorporated into seven DM scenarios.</div></div><div><h3>Results</h3><div>The study led to the creation of a Decision Mining Method (DMM), which includes 138 method fragments grouped into eleven categories. These fragments were systematically merged to form a comprehensive DMM. The method encapsulates the complexity of DM and provides practical applicability in real-world scenarios, highlighted by the identification of seven distinct scenarios in DM phases. The study also conducted the first SLR in the DM field, providing a comprehensive overview of current practices and outcomes.</div></div><div><h3>Conclusion</h3><div>The study helps in advancing the DM field by creating a structured approach and a comprehensive method for DM, aligning with recent developments in the field. It successfully aggregated the fragmented DM domain into a cohesive methodological overview, crucial for future research. The study also lays out a detailed agenda for future research, focusing on expanding and validating the DMM, incorporating cross-disciplinary insights, and addressing the challenges in machine learning within DM. The future research directions aim to refine and broaden the applicability of the DMM, ensuring its effectiveness in diverse practical contexts and contributing to a more holistic and comprehensive approach to decision mining.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107627"},"PeriodicalIF":3.8,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142722028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Systematic mapping study on requirements engineering for regulatory compliance of software systems 关于软件系统合规性要求工程的系统映射研究
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-10 DOI: 10.1016/j.infsof.2024.107622
Oleksandr Kosenkov , Parisa Elahidoost , Tony Gorschek , Jannik Fischbach , Daniel Mendez , Michael Unterkalmsteiner , Davide Fucci , Rahul Mohanani
{"title":"Systematic mapping study on requirements engineering for regulatory compliance of software systems","authors":"Oleksandr Kosenkov ,&nbsp;Parisa Elahidoost ,&nbsp;Tony Gorschek ,&nbsp;Jannik Fischbach ,&nbsp;Daniel Mendez ,&nbsp;Michael Unterkalmsteiner ,&nbsp;Davide Fucci ,&nbsp;Rahul Mohanani","doi":"10.1016/j.infsof.2024.107622","DOIUrl":"10.1016/j.infsof.2024.107622","url":null,"abstract":"<div><h3>Context:</h3><div>As the diversity and complexity of regulations affecting Software-Intensive Products and Services (SIPS) is increasing, software engineers need to address the growing regulatory scrutiny. We argue that, as with any other non-negotiable requirements, SIPS compliance should be addressed early in SIPS engineering—i.e., during requirements engineering (RE).</div></div><div><h3>Objectives:</h3><div>In the conditions of the expanding regulatory landscape, existing research offers scattered insights into regulatory compliance of SIPS. This study addresses the pressing need for a structured overview of the state of the art in software RE and its contribution to regulatory compliance of SIPS.</div></div><div><h3>Method:</h3><div>We conducted a systematic mapping study to provide an overview of the current state of research regarding challenges, principles, and practices for regulatory compliance of SIPS related to RE. We focused on the role of RE and its contribution to other SIPS lifecycle process areas. We retrieved 6914 studies published from 2017 (January 1) until 2023 (December 31) from four academic databases, which we filtered down to 280 relevant primary studies.</div></div><div><h3>Results:</h3><div>We identified and categorized the RE-related challenges in regulatory compliance of SIPS and their potential connection to six types of principles and practices addressing challenges. We found that about 13.6% of the primary studies considered the involvement of both software engineers and legal experts in developing principles and practices. About 20.7% of primary studies considered RE in connection to other process areas. Most primary studies focused on a few popular regulation fields (privacy, quality) and application domains (healthcare, software development, avionics). Our results suggest that there can be differences in terms of challenges and involvement of stakeholders across different fields of regulation.</div></div><div><h3>Conclusion:</h3><div>Our findings highlight the need for an in-depth investigation of stakeholders’ roles, relationships between process areas, and specific challenges for distinct regulatory fields to guide research and practice.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107622"},"PeriodicalIF":3.8,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142698416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A software product line approach for developing hybrid software systems 开发混合软件系统的软件生产线方法
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-10 DOI: 10.1016/j.infsof.2024.107625
Samuel Sepúlveda , Ricardo Pérez-Castillo , Mario Piattini
{"title":"A software product line approach for developing hybrid software systems","authors":"Samuel Sepúlveda ,&nbsp;Ricardo Pérez-Castillo ,&nbsp;Mario Piattini","doi":"10.1016/j.infsof.2024.107625","DOIUrl":"10.1016/j.infsof.2024.107625","url":null,"abstract":"<div><h3>Context:</h3><div>Quantum computing is rapidly emerging as a transformative force in technology. We will soon increasingly encounter hybrid systems that combine quantum technology with classical software. Software engineering techniques will be required to manage the complexity of designing such systems and their reuse.</div></div><div><h3>Objective:</h3><div>This paper introduces preliminary ideas concerning developing quantum–classical software using a Software Product Line approach.</div></div><div><h3>Method:</h3><div>This approach addresses the mentioned challenges and provides a feature model and a whole process to manage variability during the design and development of hybrid quantum–classical software. The usage of this approach is illustrated and discussed using an example in the logistics domain.</div></div><div><h3>Results:</h3><div>The preliminary insights show the feasibility and suitability of applying the proposed approach to develop complex quantum–classical software.</div></div><div><h3>Conclusions:</h3><div>The main implication of this research is that it can help to manage complexity, maximize the reuse of classical and quantum software components, and deal with the highly changing technological stack in the current quantum computing field.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107625"},"PeriodicalIF":3.8,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142656358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluating the understandability and user acceptance of Attack-Defense Trees: Original experiment and replication 评估攻击防御树的可理解性和用户接受度:原始实验和复制
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-09 DOI: 10.1016/j.infsof.2024.107624
Giovanna Broccia , Maurice H. ter Beek , Alberto Lluch Lafuente , Paola Spoletini , Alessandro Fantechi , Alessio Ferrari
{"title":"Evaluating the understandability and user acceptance of Attack-Defense Trees: Original experiment and replication","authors":"Giovanna Broccia ,&nbsp;Maurice H. ter Beek ,&nbsp;Alberto Lluch Lafuente ,&nbsp;Paola Spoletini ,&nbsp;Alessandro Fantechi ,&nbsp;Alessio Ferrari","doi":"10.1016/j.infsof.2024.107624","DOIUrl":"10.1016/j.infsof.2024.107624","url":null,"abstract":"<div><h3>Context:</h3><div>Attack-Defense Trees (ADTs) are a graphical notation used to model and evaluate security requirements. ADTs are popular because they facilitate communication among different stakeholders involved in system security evaluation and are formal enough to be verified using methods like model checking. The understandability and user-friendliness of ADTs are claimed as key factors in their success, but these aspects, along with user acceptance, have not been evaluated empirically.</div></div><div><h3>Objectives:</h3><div>This paper presents an experiment with 25 subjects designed to assess the understandability and user acceptance of the ADT notation, along with an internal replication involving 49 subjects.</div></div><div><h3>Methods:</h3><div>The experiments adapt the Method Evaluation Model (MEM) to examine understandability variables (i.e., effectiveness and efficiency in using ADTs) and user acceptance variables (i.e., ease of use, usefulness, and intention to use). The MEM is also used to evaluate the relationships between these dimensions. In addition, a comparative analysis of the results of the two experiments is carried out.</div></div><div><h3>Results:</h3><div>With some minor differences, the outcomes of the two experiments are aligned. The results demonstrate that ADTs are well understood by participants, with values of understandability variables significantly above established thresholds. They are also highly appreciated, particularly for their ease of use. The results also show that users who are more effective in using the notation tend to evaluate it better in terms of usefulness.</div></div><div><h3>Conclusion:</h3><div>These studies provide empirical evidence supporting both the understandability and perceived acceptance of ADTs, thus encouraging further adoption of the notation in industrial contexts, and development of supporting tools.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107624"},"PeriodicalIF":3.8,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142656359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Who uses personas in requirements engineering: The practitioners’ perspective 谁在需求工程中使用角色?实践者的视角
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-09 DOI: 10.1016/j.infsof.2024.107609
Yi Wang , Chetan Arora , Xiao Liu , Thuong Hoang , Vasudha Malhotra , Ben Cheng , John Grundy
{"title":"Who uses personas in requirements engineering: The practitioners’ perspective","authors":"Yi Wang ,&nbsp;Chetan Arora ,&nbsp;Xiao Liu ,&nbsp;Thuong Hoang ,&nbsp;Vasudha Malhotra ,&nbsp;Ben Cheng ,&nbsp;John Grundy","doi":"10.1016/j.infsof.2024.107609","DOIUrl":"10.1016/j.infsof.2024.107609","url":null,"abstract":"<div><h3>Context:</h3><div>Personas are commonly employed in software projects to better understand end-users needs. Despite their frequent usage, there is a limited understanding of their practical application and effectiveness.</div></div><div><h3>Objective:</h3><div>This paper aims to investigate the current practices, methods, and challenges associated with using personas in software development.</div></div><div><h3>Methods:</h3><div>A two-step investigation was conducted, comprising interviews with 26 software developers, UI/UX designers, business analysts, and product managers, along with a survey of 203 practitioners.</div></div><div><h3>Results:</h3><div>The findings reveal variations in the frequency and effectiveness of personas across different software projects and IT companies. Additionally, the study highlights the challenges practitioners face when using personas and the reasons for not using them. Notably, the research shows that some human aspects (e.g., the needs of users with disabilities), often assumed to be a key feature of personas, are frequently not considered for various reasons in requirements engineering.</div></div><div><h3>Conclusions:</h3><div>The study provides actionable insights for practitioners to overcome challenges in using personas during the requirements engineering stages. Furthermore, it identifies areas for future research to enhance the effectiveness of personas in software development.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107609"},"PeriodicalIF":3.8,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142698417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
On the road to interactive LLM-based systematic mapping studies 基于 LLM 的互动式系统制图研究之路
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-10-31 DOI: 10.1016/j.infsof.2024.107611
Kai Petersen , Jan M. Gerken
{"title":"On the road to interactive LLM-based systematic mapping studies","authors":"Kai Petersen ,&nbsp;Jan M. Gerken","doi":"10.1016/j.infsof.2024.107611","DOIUrl":"10.1016/j.infsof.2024.107611","url":null,"abstract":"<div><h3>Context:</h3><div>The research volume is continuously increasing. Manual analysis of large topic scopes and continuously updating literature studies with the newest research results is effort intensive and, therefore, difficult to achieve.</div></div><div><h3>Objective:</h3><div>To discuss possibilities and next steps for using LLMs (e.g., GPT-4) in the mapping study process.</div></div><div><h3>Method:</h3><div>The research can be classified as a solution proposal. The solution was iteratively designed and discussed among the authors based on their experience with LLMs and literature reviews.</div></div><div><h3>Results:</h3><div>We propose strategies for the mapping process, outlining the use of agents and prompting strategies for each step.</div></div><div><h3>Conclusion:</h3><div>Given the potential of LLMs in literature studies, we should work on a holistic solutions for LLM-supported mapping studies.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107611"},"PeriodicalIF":3.8,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Top-down: A better strategy for incremental covering array generation 自上而下:增量覆盖阵列生成的更好策略
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-10-30 DOI: 10.1016/j.infsof.2024.107601
Yan Wang , Xintao Niu , Huayao Wu , Changhai Nie , Lei Yu , Xiaoyin Wang , Jiaxi Xu
{"title":"Top-down: A better strategy for incremental covering array generation","authors":"Yan Wang ,&nbsp;Xintao Niu ,&nbsp;Huayao Wu ,&nbsp;Changhai Nie ,&nbsp;Lei Yu ,&nbsp;Xiaoyin Wang ,&nbsp;Jiaxi Xu","doi":"10.1016/j.infsof.2024.107601","DOIUrl":"10.1016/j.infsof.2024.107601","url":null,"abstract":"<div><h3>Context:</h3><div>The Incremental Covering Array (ICA) offers a flexible and efficient test schedule for Combinatorial Testing (CT) by enabling dynamic adjustment of test strength. Despite its importance, ICA generation has been under-explored in the CT community, resulting in limited and suboptimal existing approaches.</div></div><div><h3>Objective:</h3><div>To address this gap, we introduce a novel strategy, namely <em>Top-down</em>, for optimizing ICA generation.</div></div><div><h3>Method:</h3><div>In contrast to the traditional strategy, named <em>Bottom-up</em>, <em>Top-down</em> starts with a higher-strength test set and then extracts lower-strength sets from it, thus leveraging test case generation algorithms more effectively.</div></div><div><h3>Results:</h3><div>We conducted a comparative evaluation of the two strategies across 17 real-world software with 84 total versions. The results demonstrate that compared with <em>Bottom-up</em>, the <em>Top-down</em> strategy requires less time and generates smaller ICAs while covering more higher-strength interactions. Furthermore, <em>Top-down</em> outperforms <em>Bottom-up</em> in early fault detection and code line coverage, while also surpassing the random and direct CA generation strategies.</div></div><div><h3>Conclusion:</h3><div>The <em>Top-down</em> strategy not only improved the efficiency of test case generation but also enhanced the effectiveness of fault detection in the incremental testing scenarios.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107601"},"PeriodicalIF":3.8,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586220","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A multivocal literature review on the benefits and limitations of industry-leading AutoML tools 关于业界领先的 AutoML 工具的优势和局限性的多语种文献综述
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-10-29 DOI: 10.1016/j.infsof.2024.107608
Luigi Quaranta , Kelly Azevedo , Fabio Calefato , Marcos Kalinowski
{"title":"A multivocal literature review on the benefits and limitations of industry-leading AutoML tools","authors":"Luigi Quaranta ,&nbsp;Kelly Azevedo ,&nbsp;Fabio Calefato ,&nbsp;Marcos Kalinowski","doi":"10.1016/j.infsof.2024.107608","DOIUrl":"10.1016/j.infsof.2024.107608","url":null,"abstract":"<div><h3>Context:</h3><div>Rapid advancements in Artificial Intelligence (AI) and Machine Learning (ML) are revolutionizing software engineering in every application domain, driving unprecedented transformations and fostering innovation. However, despite these advances, several organizations are experiencing friction in the adoption of ML-based technologies, mainly due to the current shortage of ML professionals. In this context, Automated Machine Learning (AutoML) techniques have been presented as a promising solution to democratize ML adoption, even in the absence of specialized people.</div></div><div><h3>Objective:</h3><div>Our research aims to provide an overview of the evidence on the benefits and limitations of AutoML tools being adopted in industry.</div></div><div><h3>Methods:</h3><div>We conducted a Multivocal Literature Review, which allowed us to identify 54 sources from the academic literature and 108 sources from the grey literature reporting on AutoML benefits and limitations. We extracted explicitly reported benefits and limitations from the papers and applied the thematic analysis method for synthesis.</div></div><div><h3>Results:</h3><div>In general, we identified 18 reported benefits and 25 limitations. Concerning the benefits, we highlight that AutoML tools can help streamline the core steps of ML workflows, namely data preparation, feature engineering, model construction, and hyperparameter tuning—with concrete benefits on model performance, efficiency, and scalability. In addition, AutoML empowers both novice and experienced data scientists, promoting ML accessibility. However, we highlight several limitations that may represent obstacles to the widespread adoption of AutoML. For instance, AutoML tools may introduce barriers to transparency and interoperability, exhibit limited flexibility for complex scenarios, and offer inconsistent coverage of the ML workflow.</div></div><div><h3>Conclusion:</h3><div>The effectiveness of AutoML in facilitating the adoption of machine learning by users may vary depending on the specific tool and the context in which it is used. Today, AutoML tools are used to increase human expertise rather than replace it and, as such, require skilled users.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"178 ","pages":"Article 107608"},"PeriodicalIF":3.8,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142553475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信