Information and Software Technology最新文献

筛选
英文 中文
Locating requirements in backlog items: Content analysis and experiments with large language models
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-12-12 DOI: 10.1016/j.infsof.2024.107644
Ashley T. van Can, Fabiano Dalpiaz
{"title":"Locating requirements in backlog items: Content analysis and experiments with large language models","authors":"Ashley T. van Can,&nbsp;Fabiano Dalpiaz","doi":"10.1016/j.infsof.2024.107644","DOIUrl":"10.1016/j.infsof.2024.107644","url":null,"abstract":"<div><h3>Context:</h3><div>As agile development has become mainstream, requirements are increasingly managed via issue tracking systems (ITSs). These systems provide a single point of access to the product and sprint backlogs, bugs, ideas, and tasks for the development team. ITSs do not clearly separate requirements from work items.</div></div><div><h3>Objective:</h3><div>We first tackle a <em>knowledge problem</em> concerning how requirements are formulated in ITSs, including their categorization and granularity, the presence of multiple requirements, and the existence of a motivation. Second, to assist practitioners in finding requirements in poorly organized ITSs without changing their way of working, we investigate the potential of automated techniques for identifying and classifying requirements in backlog items.</div></div><div><h3>Method:</h3><div>Through quantitative content analysis, we analyze 1,636 product backlog items sampled from fourteen projects. To explore automated techniques for identifying requirements, we experiment with large language models (LLMs) due to their recent significance in NLP.</div></div><div><h3>Results:</h3><div>The labeling of backlog items is largely inconsistent, and user-oriented functional requirements are the prevalent category. A backlog item often contains multiple requirements with different levels of granularity. The experiments with LLMs reveal that encoder-only models (BERT and RoBERTa) are most suitable for extracting and classifying requirements in backlog items compared to decoder-only models (Llama 3, Mistral 7B and ChatGPT with GPT 4).</div></div><div><h3>Conclusion:</h3><div>We reveal knowledge and patterns about requirements documentation in ITSs, leading to a better empirical understanding of Agile RE. The experimental results with LLMs provide the foundation for developing automated, unobtrusive tools that identify and classify requirements in ITSs.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107644"},"PeriodicalIF":3.8,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143091909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Exploring the impact of feedback on remote SW development teams
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-12-10 DOI: 10.1016/j.infsof.2024.107649
Ana Beatriz Cavalcanti , Carina Alves , João Araújo
{"title":"Exploring the impact of feedback on remote SW development teams","authors":"Ana Beatriz Cavalcanti ,&nbsp;Carina Alves ,&nbsp;João Araújo","doi":"10.1016/j.infsof.2024.107649","DOIUrl":"10.1016/j.infsof.2024.107649","url":null,"abstract":"<div><h3>Context</h3><div>Feedback is essential in the routine of software development teams. It provides information on professionals’ performance, align goals, and manage conflicts. The growing adoption of the remote work model has created new challenges for the effective use of feedback by organizations. Despite the recognized importance of feedback, few studies focused on understanding how feedback practices are currently conducted in remote software development teams.</div></div><div><h3>Objective</h3><div>This work aims to explore the impact of feedback on software development teams working remotely. In addition, we aim to provide valuable insights on how teams can optimize the outcomes of feedback practices.</div></div><div><h3>Method</h3><div>We adopted a mixed-method approach to investigate how feedback practices are conducted in remote software development teams. We performed a multivocal literature review to map the benefits, challenges, and good practices mentioned in the literature. Then, we conducted semi-structured interviews with 10 leaders and managers to understand their perceptions about feedback practices. Finally, we surveyed 83 team members to understand their perceptions and feelings about receiving feedback.</div></div><div><h3>Results</h3><div>We found out that the key benefits of feedback include a boost in individual engagement and team performance. In contrast, common challenges involved in remote feedback are communication gaps due to the adoption of digital channels and difficulty in providing and receiving negative feedback. Finally, our study proposes good practices to improve the feedback outcomes, such as: using multidimensional indicators to evaluate team members, providing a tangible goal-oriented development plan, and adopting continuous feedback follow-up.</div></div><div><h3>Conclusion</h3><div>We synthesized evidence from multiple sources by adopting three research methods to understand the effects of feedback on remote software development teams. Finally, we provided a set of actionable insights on how to optimize the feedback on remote software development teams.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107649"},"PeriodicalIF":3.8,"publicationDate":"2024-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143092335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Service engineering for quantum computing: Ensuring high-quality quantum services
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-12-09 DOI: 10.1016/j.infsof.2024.107643
Ana Díaz , Jaime Alvarado-Valiente , Javier Romero-Álvarez , Enrique Moguel , Jose Garcia-Alonso , Moisés Rodríguez , Ignacio García-Rodríguez , Juan M. Murillo
{"title":"Service engineering for quantum computing: Ensuring high-quality quantum services","authors":"Ana Díaz ,&nbsp;Jaime Alvarado-Valiente ,&nbsp;Javier Romero-Álvarez ,&nbsp;Enrique Moguel ,&nbsp;Jose Garcia-Alonso ,&nbsp;Moisés Rodríguez ,&nbsp;Ignacio García-Rodríguez ,&nbsp;Juan M. Murillo","doi":"10.1016/j.infsof.2024.107643","DOIUrl":"10.1016/j.infsof.2024.107643","url":null,"abstract":"<div><h3>Context:</h3><div>Quantum computing is transforming the world and driving advanced applications in fields such as healthcare and economics. However, ensuring high-quality quantum software remains critical to its adoption across the industry. As quantum technology moves closer to practical applications, it faces significant challenges. Developers face platform-dependent complexities that make the creation of quantum applications a time-consuming process. In addition, the lack of mature tools further hampers progress and can compromise the quality of service.</div></div><div><h3>Objective:</h3><div>The objective of this paper is to address the pressing need for quantum software quality assurance, presenting a solution for defining and using quantum services, by employing classical service engineering techniques and methods.</div></div><div><h3>Methods:</h3><div>A process is presented for improving the generation, deployment, and quality assessment of quantum services using an extended OpenAPI Specification and the SonarQube tool. This process also integrates the automatic generation of code for the IBM Quantum provider and its deployment in containers ready for user consumption.</div></div><div><h3>Results:</h3><div>After a detailed and individualized evaluation of the 40 implementations of quantum algorithms using the developed environment, the results reveal significant variability in the analyzability of the algorithms. This will serve in the future as a reference and guide for the continuous improvement of quantum algorithms in terms of their performance and efficiency in solving complex problems in various quantum application areas.</div></div><div><h3>Conclusions:</h3><div>This research offers a fundamental contribution to the evolution of quantum computing by introducing a comprehensive framework for quantum software quality assurance. The proposed approach not only addresses some of the existing problems in quantum software, but also paves the way for the development of quantum algorithms and their servitization.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107643"},"PeriodicalIF":3.8,"publicationDate":"2024-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143091908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Improving bug triage with the bug personalized tossing relationship
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-12-09 DOI: 10.1016/j.infsof.2024.107642
Wei Wei , Haojie Li , Xinshuang Ren , Feng Jiang , Xu Yu , Xingyu Gao , Junwei Du
{"title":"Improving bug triage with the bug personalized tossing relationship","authors":"Wei Wei ,&nbsp;Haojie Li ,&nbsp;Xinshuang Ren ,&nbsp;Feng Jiang ,&nbsp;Xu Yu ,&nbsp;Xingyu Gao ,&nbsp;Junwei Du","doi":"10.1016/j.infsof.2024.107642","DOIUrl":"10.1016/j.infsof.2024.107642","url":null,"abstract":"<div><h3>Background:</h3><div>In open-source software projects, the main task of bug triage is accurately assigning bugs to appropriate developers. Statistics indicate that about 50% of bugs are reassigned (also called “tossed”) at least once, greatly extending the time for bug fixing. Research studies have shown that combining historical tossing relationships can significantly improve bug triage performance.</div></div><div><h3>Objective:</h3><div>The current research on utilizing bug tossing relationships can be mainly divided into two categories: (1) During the reassignment phase, only developers with the highest probability of tossing relationships are selected. (2) Use attribute filtering mechanism to filter and match developers. However, these approaches fail to fully consider the matching degree between developers’ abilities and the knowledge required to fix current bugs. We are attempting to propose an approach to address the above problem.</div></div><div><h3>Approach:</h3><div>We propose an approach to improve bug triage with the Bug Personalized Tossing Relationship (BPTRM). It uses a tossing transition probability matrix derived from historical tossing paths to help recommend suitable developers for solving bug reports.</div></div><div><h3>Result:</h3><div>Experimental results from various data sets within Eclipse and Mozilla indicate that BPTRM improves average recommendation performance by at least 14.38% compared to different initial assignment approaches. In addition, compared to baselines, BPTRM improves the average accuracy by 14.66% and shortens the average bug tossing length by 16.19%.</div></div><div><h3>Conclusion:</h3><div>1. The BPTRM approach, combined with personalized bug tossing relationships, precisely matches developers’ abilities and the knowledge required to fix current bugs. 2. This effectively improves the bug triage’s accuracy and shortens the bug tossing’s length.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107642"},"PeriodicalIF":3.8,"publicationDate":"2024-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143092440","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
DeepCNP: An efficient white-box testing of deep neural networks by aligning critical neuron paths
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-29 DOI: 10.1016/j.infsof.2024.107640
Weiguang Liu, Senlin Luo, Limin Pan, Zhao Zhang
{"title":"DeepCNP: An efficient white-box testing of deep neural networks by aligning critical neuron paths","authors":"Weiguang Liu,&nbsp;Senlin Luo,&nbsp;Limin Pan,&nbsp;Zhao Zhang","doi":"10.1016/j.infsof.2024.107640","DOIUrl":"10.1016/j.infsof.2024.107640","url":null,"abstract":"<div><h3>Context</h3><div>Erroneous decisions of Deep Neural Networks may pose a significant threat to Deep Learning systems deployed in security-critical domains. The key to testing DNNs is to propose a testing technique to generate test cases that can detect more defects of the models. It has been demonstrated that coverage-guided fuzz testing methods are difficult to detect the correctness defects of model's decision logic. Meanwhile, the neuron activation threshold is set based on experience, which increases the uncertainty of the test even more. In addition, the randomly selected seed mutations are prone to generate a large number of invalid test cases, which has a great impact on the testing efficiency.</div></div><div><h3>Objective</h3><div>This paper introduces DeepCNP, a method that combines Critical Neuron Paths alignment and dynamic seeds selection strategy, which can comprehensively and efficiently test all the decision paths of DNN and generate as many different classes of test cases as possible to expose misbehaviors of the model and thus finding defects.</div></div><div><h3>Method</h3><div>DeepCNP utilizes training data to construct decision paths determined by the neuron output distribution, and aligns different decision paths in order to generate test cases. Seeds that are easy to align are dynamically selected based on the decision paths to be tested, and the labeling of seed mutations is specified during the path alignment process, thus improving the efficiency of fuzz testing.</div></div><div><h3>Results</h3><div>Experimental results show that DeepCNP achieves new state-of-the-art results, pioneering the testing of all decision logics of the model through critical neuron path alignment, which greatly enhances the number of defects found, the efficiency and number of generated test cases.</div></div><div><h3>Conclusion</h3><div>DeepCNP comprehensively tests the decision logic of DNNs, efficiently generating a large number of test cases of different categories to expose model's misbehaviors and thus finding additional defects.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107640"},"PeriodicalIF":3.8,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143092441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Solutions toCybersecurity Challenges in Secure Vehicle-to-Vehicle Communications: A Multivocal Literature Review.
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-28 DOI: 10.1016/j.infsof.2024.107639
Naeem Ullah , Siffat Ullah Khan , Mahmood Niazi , Matteo Esposito , Arif Ali Khan , Jamal Abdul Nasir
{"title":"Solutions toCybersecurity Challenges in Secure Vehicle-to-Vehicle Communications: A Multivocal Literature Review.","authors":"Naeem Ullah ,&nbsp;Siffat Ullah Khan ,&nbsp;Mahmood Niazi ,&nbsp;Matteo Esposito ,&nbsp;Arif Ali Khan ,&nbsp;Jamal Abdul Nasir","doi":"10.1016/j.infsof.2024.107639","DOIUrl":"10.1016/j.infsof.2024.107639","url":null,"abstract":"<div><h3>Context</h3><div>Vehicle-to-Vehicle (V2V) technology is evolving rapidly, meeting modern transportation needs and driving economic and technological progress. V2V brings numerous benefits, enabling vehicles to communicate with each other and with infrastructure like Roadside Units (RSUs), which helps minimize collisions, reduce fatalities, and boost road safety for passengers, drivers, and pedestrians alike. Beyond safety, V2V improves traffic management and optimizes routes. However, these advancements also introduce new challenges. Greater reliance on IT makes vehicles more vulnerable to cyber-attacks and increases costs related to system installation and maintenance. This highlights a pressing need to advance V2V technology to enhance overall safety.</div></div><div><h3>Objective</h3><div>This research focuses on identifying the primary challenges and effective practices within Vehicle-to-Vehicle (V2V) communication.</div></div><div><h3>Method</h3><div>We conducted a Multivocal Literature Review (MLR) using tailored search strings derived from our research questions. This process adhered to all standard MLR steps, including protocol development, initial and final selection, quality assessment, data extraction, and synthesis.</div></div><div><h3>Results</h3><div>We have identified a list of 18 challenges in the context of V2V communication. 10 of these challenges were marked as critical challenges based on the criterion of ≥20 % occurrences in both formal and grey literature. We also identified related practices for the identified critical challenges. The identified challenges were further analyzed based on different variables such as publication periods and study strategies.</div></div><div><h3>Conclusion</h3><div>We recommend that automotive industries should prioritize addressing these challenges to enhance their readiness for secure V2V communication. Our overarching goal is to develop a Cybersecurity Challenges Mitigation Model (CCMM) based on MLR findings and industrial survey outcomes, assisting companies in the automobile sector to assess their readiness for secure V2V communication development.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107639"},"PeriodicalIF":3.8,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143092439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Native cross-platform app development using the SequalsK transpiler 使用SequalsK转译器进行本地跨平台应用程序开发
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-26 DOI: 10.1016/j.infsof.2024.107626
Dominik Schultes, Larissa Schneider, Tobias Heymann, Franziska Wild
{"title":"Native cross-platform app development using the SequalsK transpiler","authors":"Dominik Schultes,&nbsp;Larissa Schneider,&nbsp;Tobias Heymann,&nbsp;Franziska Wild","doi":"10.1016/j.infsof.2024.107626","DOIUrl":"10.1016/j.infsof.2024.107626","url":null,"abstract":"<div><h3>Context:</h3><div>Developing two separate versions of an app for iOS and Android requires significant effort. Existing cross-platform development frameworks may reduce this effort, but they also come with tradeoffs such as high tool dependency.</div></div><div><h3>Objective:</h3><div>To avoid the drawbacks of current methods, we introduce a new approach to cross-platform app development, provide the necessary tools, and conduct a thorough evaluation to demonstrate the feasibility of our proposed approach.</div></div><div><h3>Method:</h3><div>The central idea of the new <em>native cross-platform development</em> approach is to actively develop apps in both native programming languages, Kotlin for Android and Swift for iOS, while exchanging considerable parts of the source code in a bidirectional fashion using a deterministic transpiler. As the centerpiece of our proposed development approach, we present such a <em>bidirectional</em> Swift-Kotlin transpiler, called <em>SequalsK</em>. It supports the majority of the important constructs of both languages and is able to generate syntactically and semantically correct Kotlin code out of Swift code <em>and vice versa</em>.</div></div><div><h3>Results:</h3><div>In our evaluation, we determined that SequalsK is the sole existing bidirectional transpiler, distinguishing it from other transpilers that support only one direction. For the Kotlin-to-Swift direction, SequalsK emerges as the premier transpiler, while for the reverse direction, it stands among the top transpilers. Through six distinct case studies, we applied our native cross-platform development approach, showcasing its ability to fulfill all goals. Across each study, we successfully generated fully-functional native Android and iOS apps, achieving significant time savings as up to 86 percent of the source code has to be programmed only once and can be transpiled to the other involved programming language automatically.</div></div><div><h3>Conclusion:</h3><div>At the moment, in particular data structures and business logic can be transpiled successfully. In the future, we expect further improvements by extending the SequalsK transpiler in order to process user-interface parts as well.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107626"},"PeriodicalIF":3.8,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142748033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Redefining crowdsourced test report prioritization: An innovative approach with large language model 重新定义众包测试报告的优先级:一种具有大型语言模型的创新方法
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-26 DOI: 10.1016/j.infsof.2024.107629
Yuchen Ling , Shengcheng Yu , Chunrong Fang , Guobin Pan , Jun Wang , Jia Liu
{"title":"Redefining crowdsourced test report prioritization: An innovative approach with large language model","authors":"Yuchen Ling ,&nbsp;Shengcheng Yu ,&nbsp;Chunrong Fang ,&nbsp;Guobin Pan ,&nbsp;Jun Wang ,&nbsp;Jia Liu","doi":"10.1016/j.infsof.2024.107629","DOIUrl":"10.1016/j.infsof.2024.107629","url":null,"abstract":"<div><h3>Context:</h3><div>Crowdsourced testing has gained popularity in software testing, especially for mobile app testing, due to its ability to bring diversity and tackle fragmentation issues. However, the openness of crowdsourced testing presents challenges, particularly in the manual review of numerous test reports, which is time-consuming and labor-intensive.</div></div><div><h3>Objective:</h3><div>The primary goal of this research is to improve the efficiency of review processes in crowdsourced testing. Traditional approaches to test report prioritization lack a deep understanding of semantic information in textual descriptions of these reports. This paper introduces LLMPrior, a novel approach for prioritizing crowdsourced test reports using large language models (LLMs).</div></div><div><h3>Method:</h3><div>LLMPrior leverages LLMs for the analysis and clustering of crowdsourced test reports based on the types of bugs revealed in their textual descriptions. This involves using prompt engineering techniques to enhance the performance of LLMs. Following the clustering, a recurrent selection algorithm is applied to prioritize the reports.</div></div><div><h3>Results:</h3><div>Empirical experiments are conducted to evaluate the effectiveness of LLMPrior. The findings indicate that LLMPrior not only surpasses current state-of-the-art approaches in terms of performance but also proves to be more feasible, efficient, and reliable. This success is attributed to the use of prompt engineering techniques and the cluster-based prioritization strategy.</div></div><div><h3>Conclusion:</h3><div>LLMPrior represents a significant advancement in crowdsourced test report prioritization. By effectively utilizing large language models and a cluster-based strategy, it addresses the challenges in traditional prioritization approaches, offering a more efficient and reliable solution for app developers dealing with crowdsourced test reports.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107629"},"PeriodicalIF":3.8,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142748032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Markov model based coverage testing of deep learning software systems 基于马尔可夫模型的深度学习软件系统覆盖率测试
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-23 DOI: 10.1016/j.infsof.2024.107628
Ying Shi, Beibei Yin, Jing-Ao Shi
{"title":"Markov model based coverage testing of deep learning software systems","authors":"Ying Shi,&nbsp;Beibei Yin,&nbsp;Jing-Ao Shi","doi":"10.1016/j.infsof.2024.107628","DOIUrl":"10.1016/j.infsof.2024.107628","url":null,"abstract":"<div><h3>Context:</h3><div>Deep Learning (DL) software systems have been widely deployed in safety and security-critical domains, which calls for systematic testing to guarantee their accuracy and reliability. Objective measurement of test quality is one of the key issues in software testing. Recently, many coverage criteria have been proposed to measure the testing adequacy of Deep Neural Networks (DNNs).</div></div><div><h3>Objective:</h3><div>Recent research demonstrates that existing criteria have some limitations on interpreting the increasingly diverse behaviors of DNNs or clarifying the relationship between the coverage and the decision logic of DNNs. Moreover, some evaluations argue against the correlation between coverage and defect detection. In this paper, a novel coverage approach is proposed to interpret the internal information of programs.</div></div><div><h3>Methods:</h3><div>The process of coverage testing is formalized and quantified by constructing Markov models based on critical neurons extracted using Layer-wise Relevance Propagation in the structure of DNNs. The difference in the transition matrix of Markov chains between training and testing data is measured by KL divergence, and it is developed as a coverage criterion.</div></div><div><h3>Results:</h3><div>The values of the proposed coverage increase as the number of classes increases. The values are different for various test suites, and they become higher with the addition of new samples. Higher coverage values are observed to correlate with an increased fault detection capability.</div></div><div><h3>Conclusion:</h3><div>The experimental results illustrate that the proposed approach can effectively measure actual diversity and exhibit more adaptability to additional test cases. Furthermore, there is a positive correlation between the proposed coverage and fault detection, which provides support for test case selection guided by coverage.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107628"},"PeriodicalIF":3.8,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142722027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An alternative to code comment generation? Generating comment from bytecode 代码注释生成的替代方案?从字节码生成注释
IF 3.8 2区 计算机科学
Information and Software Technology Pub Date : 2024-11-22 DOI: 10.1016/j.infsof.2024.107623
Xiangping Chen , Junqi Chen , Zhilu Lian , Yuan Huang , Xiaocong Zhou , Yunzhi Wu , Zibin Zheng
{"title":"An alternative to code comment generation? Generating comment from bytecode","authors":"Xiangping Chen ,&nbsp;Junqi Chen ,&nbsp;Zhilu Lian ,&nbsp;Yuan Huang ,&nbsp;Xiaocong Zhou ,&nbsp;Yunzhi Wu ,&nbsp;Zibin Zheng","doi":"10.1016/j.infsof.2024.107623","DOIUrl":"10.1016/j.infsof.2024.107623","url":null,"abstract":"<div><h3>Context:</h3><div>Due to the importance and necessity of code comments, recent works propose many comment generation models with source code as input. But sometimes there has no access to obtain the source code, only the bytecode, such as many Apps.</div></div><div><h3>Objective:</h3><div>If there is a way to generate comments for bytecode directly, tasks such as malware detection and understanding closed-source software can benefit from the generated comment because it improves the understandability of the system. Therefore, we propose a novel approach called ByteGen to generate comments from bytecode.</div></div><div><h3>Methods:</h3><div>Specifically, to extract the structure characteristic of the bytecode, we utilize the control flow graph (CFG) of the bytecode and use a special traversal named enhanced SBT to serialize CFG. The enhanced SBT can completely preserve the structure of the CFG in a sequence. We set up experiments on a dataset with a scale of about 50,000 bytecode-comment pairs collected from Maven.</div></div><div><h3>Results:</h3><div>Experimental results show that the average BLEU-4 score of ByteGen is 28.67, which outperforms several baselines, and a human study also indicates the effectiveness of ByteGen in generating comments from bytecodes.</div></div><div><h3>Conclusion:</h3><div>In general, ByteGen performs better than other baselines. Therefore, this also proves the effectiveness of our approach in the code comment generation scenario without source code.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"179 ","pages":"Article 107623"},"PeriodicalIF":3.8,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142748030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信