Journal of Software-Evolution and Process最新文献

筛选
英文 中文
On the value of instance selection for bug resolution prediction performance 关于实例选择对错误解决预测性能的价值
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-07-02 DOI: 10.1002/smr.2710
Chaymae Miloudi, Laila Cheikhi, Ali Idri, Alain Abran
{"title":"On the value of instance selection for bug resolution prediction performance","authors":"Chaymae Miloudi,&nbsp;Laila Cheikhi,&nbsp;Ali Idri,&nbsp;Alain Abran","doi":"10.1002/smr.2710","DOIUrl":"10.1002/smr.2710","url":null,"abstract":"<p>Software maintenance is a challenging and laborious software management activity, especially for open-source software. The bugs reports of such software allow tracking maintenance activities and were used in several empirical studies to better predict the bug resolution effort. These reports are known for their large size and contain nonrelevant instances that need to be preprocessed to be suitable for use. To this end, instance selection (IS) has been proposed in the literature as a way to reduce the size of the datasets, while keeping the relevant instances. The objective of this study is to perform an empirical study that investigates the impact of data preprocessing through IS on the performance of bug resolution prediction classifiers. To deal with this, four IS algorithms, namely, edited nearest neighbor (ENN), repeated ENN, all-k nearest neighbors, and model class selection, are applied on five large datasets, together with five machine learning techniques. Overall, 125 experiments were performed and compared. The findings of this study highlight the positive impact of IS in providing better estimates for bug resolution prediction classifiers, in particular using repeated ENN and ENN algorithms.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 11","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141552701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Trustworthy and collaborative traceability management: Experts’ feedback on a blockchain-enabled framework 值得信赖的协作式溯源管理:专家对区块链框架的反馈意见
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-07-01 DOI: 10.1002/smr.2707
Selina Demi, Mary Sánchez-Gordón, Monica Kristiansen, Xabier Larrucea
{"title":"Trustworthy and collaborative traceability management: Experts’ feedback on a blockchain-enabled framework","authors":"Selina Demi,&nbsp;Mary Sánchez-Gordón,&nbsp;Monica Kristiansen,&nbsp;Xabier Larrucea","doi":"10.1002/smr.2707","DOIUrl":"10.1002/smr.2707","url":null,"abstract":"<p>Blockchain technology has attracted significant attention in both academia and industry. Recently, the application of blockchain has been advocated in software engineering. The global software engineering paradigm exacerbates trust issues, as distributed and cross-organizational teams need to share software artifacts. In such a context, there is a need for a decentralized yet reliable traceability knowledge base to keep track of what/how/when/by whom software artifacts were created or changed. This study presents a blockchain-enabled framework for trustworthy and collaborative traceability management and identifies benefits, challenges, and potential improvements based on the feedback of software engineering experts. A qualitative approach was followed in this study through semistructured interviews with software engineering (SE) experts. Transcripts were analyzed by applying the content analysis technique. The results indicated the emergence of five categories, further grouped into three main categories: experts' perceptions, blockchain-based software process improvement, and experts' recommendations. In addition, the findings suggested four archetypes of organizations that may be interested in blockchain technology: distributed organizations, organizations with contract-based projects, organizations in regulated domains, and regulators who may push the use of this technology. Further efforts should be devoted to the integration of the proposal with tools used throughout the software development lifecycle and leveraging the potential of smart contracts in validating the implementation of requirements automatically.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 11","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/smr.2707","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141514087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Software maintenance practices using agile methods towards cloud environment: A systematic mapping 面向云环境使用敏捷方法的软件维护实践:系统映射
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-07-01 DOI: 10.1002/smr.2698
Mohammed Almashhadani, Alok Mishra, Ali Yazici
{"title":"Software maintenance practices using agile methods towards cloud environment: A systematic mapping","authors":"Mohammed Almashhadani,&nbsp;Alok Mishra,&nbsp;Ali Yazici","doi":"10.1002/smr.2698","DOIUrl":"10.1002/smr.2698","url":null,"abstract":"<p>Agile methods have emerged to overcome the obstacles of structured methodologies, such as the waterfall, prototype, spiral, and so on. There are studies showing the usefulness of agile approaches in software development. However, studies on Agile maintenance are very limited in number. Regardless of the chosen methodology, software maintenance can be carried out in either a local (on-the-premise) or global (distributed) environment. In a local environment, the software maintenance team is co-located on the same premises, while in a global environment, the team is geographically dispersed from the customer. The main objective of this Systematic Mapping (SM) study is to identify the practices useful for software maintenance using the Agile approaches in the Cloud environment. We have conducted a comprehensive search in well-known digital databases and examined the articles that map to the pre-defined inclusion criteria. The study selected and analyzed 48 articles out of 320 published between 2000 and 2022. The findings of the mapping study reveal that Agile can resolve the major issues faced in traditional software maintenance, making the role of this approach significant in global/distributed software maintenance. Cloud computing plays a vital role in software maintenance. Most of the studies highlight the application of XP- and Scrum-based Agile maintenance models. The study found a need for more Agile maintenance solutions in the cloud, highlighting the importance of agile in software maintenance, both locally and globally. Irrespective of the environment, Cloud computing provides a centralized platform for collaboration and communication, while also offering scalability and flexibility to adapt to diverse infrastructure needs. This allows agile maintenance practices to be implemented across both local and global environments, leveraging the cloud's capabilities to overcome geographical and infrastructural challenges.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 11","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/smr.2698","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141514086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Towards a security-optimized approach for the microservice-oriented decomposition 面向微服务分解的安全优化方法
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-27 DOI: 10.1002/smr.2670
Xiaodong Liu, Zhikun Chen, Yu Qian, Chenxing Zhong, Huang Huang, Shanshan Li, Dong Shao
{"title":"Towards a security-optimized approach for the microservice-oriented decomposition","authors":"Xiaodong Liu,&nbsp;Zhikun Chen,&nbsp;Yu Qian,&nbsp;Chenxing Zhong,&nbsp;Huang Huang,&nbsp;Shanshan Li,&nbsp;Dong Shao","doi":"10.1002/smr.2670","DOIUrl":"10.1002/smr.2670","url":null,"abstract":"<p>Microservice architecture (MSA) is a mainstream architectural style due to its high maintainability and scalability. In practice, an appropriate microservice-oriented decomposition is the foundation to make a system enjoy the benefits of MSA. In terms of decomposing monolithic systems into microservices, researchers have been exploring many optimization objectives, of which modularity is a predominantly focused quality attribute. Security is also a critical quality attribute, that measures the extent to which a system protects data from malicious access or use by attackers. Considering security in microservices-oriented decomposition can help avoid the risk of leaking critical data and other unexpected software security issues. However, few researchers consider the security objective during microservice-oriented decomposition, because the measurement of security and the trade-off with other objectives are challenging in reality. To bridge this research gap, we propose a security-optimized approach for microservice-oriented decomposition (So4MoD). In this approach, we adapt five metrics from previous studies for the measurement of the data security of candidate microservices. A multi-objective optimization algorithm based on NSGA-II is designed to search for microservices with optimized security and modularity. To validate the effectiveness of the proposed So4MoD, we perform several experiments on eight open-source projects and compare the decomposition results to other three state-of-the-art approaches, that is, FoSCI, CO-GCN, and MSExtractor. The experiment results show that our approach can achieve at least an 11.5% improvement in terms of security metrics. Moreover, the decomposition results of So4MoD outperform other approaches in four modularity metrics, demonstrating that So4MoD can optimize data security while pursuing a well-modularized MSA.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 10","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506978","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Crowdsourced bug report severity prediction based on text and image understanding via heterogeneous graph convolutional networks 基于异构图卷积网络的文本和图像理解的众包错误报告严重性预测
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-27 DOI: 10.1002/smr.2705
Yifan Wu, Chendong Lin, An Liu, Lei Zhao, Xiaofang Zhang
{"title":"Crowdsourced bug report severity prediction based on text and image understanding via heterogeneous graph convolutional networks","authors":"Yifan Wu,&nbsp;Chendong Lin,&nbsp;An Liu,&nbsp;Lei Zhao,&nbsp;Xiaofang Zhang","doi":"10.1002/smr.2705","DOIUrl":"10.1002/smr.2705","url":null,"abstract":"<p>In the process of crowdsourced testing, massive bug reports are submitted. Among them, the severity level of the bug report is an important indicator for traigers of crowdsourced platforms to arrange the order of reports efficiently so that developers can prioritize high-severity defects. A lot of work has been devoted to the study of automatically assigning severity levels to a large number of bug reports in crowdsourcing test systems. The research objects of these works are standard bug reports, focusing on the text part of the report, using various feature engineering methods and classification techniques. However, while achieving good performance, these methods still need to overcome two challenges: no consideration of image information in mobile testing and discontinuous semantic information of words in bug reports. In this paper, we propose a new method of severity prediction by using heterogeneous graph convolutional networks with screenshots (SPHGCN-S), which combines text features and screenshots information to understand the report more comprehensively. In addition, our approach applies the heterogeneous graph convolutional network (HGCN) architecture, which can capture the global word information to alleviate the semantic problem of word discontinuity and underlying relations between reports. We conduct a comprehensive study to compare seven commonly adopted bug report severity prediction methods with our approach. The experimental results show that our approach SPHGCN-S can improve severity prediction performance and effectively predict reports with high severity.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 11","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141507010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluation of the fixed-point iteration of minimizing delta debugging 对最小化三角调试的定点迭代进行评估
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-23 DOI: 10.1002/smr.2702
Dániel Vince, Ákos Kiss
{"title":"Evaluation of the fixed-point iteration of minimizing delta debugging","authors":"Dániel Vince,&nbsp;Ákos Kiss","doi":"10.1002/smr.2702","DOIUrl":"10.1002/smr.2702","url":null,"abstract":"<p>The minimizing Delta Debugging (DDMIN) was among the first algorithms designed to automate the task of reducing test cases. Its popularity is based on the characteristics that it works on any kind of input, without knowledge about the input structure. Several studies proved that smaller outputs can be produced faster with more advanced techniques (e.g., building a tree representation of the input and reducing that data structure); however, if the structure is unknown or changing frequently, maintaining the descriptors might not be resource-efficient. Therefore, in this paper, we focus on the evaluation of the novel fixed-point iteration of minimizing Delta Debugging (DDMIN*) on publicly available test suites related to software engineering. Our experiments show that DDMIN* can help reduce inputs further by 48.08% on average compared to DDMIN (using lines as the units of the reduction). Although the effectiveness of the algorithm improved, it comes with the cost of additional testing steps. This study shows how the characteristics of the input affect the results and when it pays off using DDMIN*.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 10","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Automated construction of reference model for software remodularization through software evolution 通过软件进化自动构建软件重模块化参考模型
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-19 DOI: 10.1002/smr.2700
Fanyi Meng, Hai Yu, Chun Yong Chong, Ying Wang, Zhiliang Zhu
{"title":"Automated construction of reference model for software remodularization through software evolution","authors":"Fanyi Meng,&nbsp;Hai Yu,&nbsp;Chun Yong Chong,&nbsp;Ying Wang,&nbsp;Zhiliang Zhu","doi":"10.1002/smr.2700","DOIUrl":"https://doi.org/10.1002/smr.2700","url":null,"abstract":"<p>The undocumented evolution of a software project and its underlying architecture underscores the need to recover the architecture from the software's implementation-level artifacts. Despite the existence of various software remodularization techniques, they often suffer from inaccuracies, and evaluating their effectiveness is challenging due to the absence of accurate “ground-truth” architectures or reference models. Prior studies on reference model construction are time-consuming and labor-intensive as it heavily relies on manual analysis by domain experts. Besides, other existing approaches that directly utilize the directory or package structure of the latest version can be unreliable, lacking in-depth analysis of the employed software structure. To address the above limitations, in this paper, we propose <b><span>A</span></b>utomated <b><span>C</span></b>onstruction of <b><span>R</span></b>eference <b><span>M</span></b>odel (ACRM), an approach for automatically constructing reference models by assigning weights to classes for various software projects using the metadata of all software versions and historical maintenance records. We evaluate ACRM through both quantitative and qualitative analyses. The experiment results provide quantitative validation and show that the generated reference models are reasonable, as confirmed by the relationship between proposed reference models and architectural smells or bugs. Furthermore, we conduct a survey among the practitioners from industry, to gain insights from practitioners' practices and further validate the generated reference models. The survey shows that, on average, 87% of the participants agree with the reference models generated by ACRM. Moreover, we propose an improved metric, <i>wc2c</i>, which analyzes the strengths and weaknesses of different types of software clustering techniques using the proposed reference models of the given software. Finally, we discuss the potential benefits of using ACRM in analyzed projects, particularly in terms of improving software quality, reducing maintenance costs, and enhancing developer productivity.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 10","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Two sides of the same coin: A study on developers' perception of defects 一枚硬币的两面:关于开发人员对缺陷的看法的研究
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-18 DOI: 10.1002/smr.2699
Geanderson Santos, Igor Muzetti, Eduardo Figueiredo
{"title":"Two sides of the same coin: A study on developers' perception of defects","authors":"Geanderson Santos,&nbsp;Igor Muzetti,&nbsp;Eduardo Figueiredo","doi":"10.1002/smr.2699","DOIUrl":"https://doi.org/10.1002/smr.2699","url":null,"abstract":"<div>\u0000 \u0000 <p>Software defect prediction is a subject of study involving the interplay of software engineering and machine learning. The current literature proposed numerous machine learning models to predict software defects from software data, such as commits and code metrics. Further, the most recent literature employs explainability techniques to understand why machine learning models made such predictions (i.e., predicting the likelihood of a defect). As a result, developers are expected to reason on the software features that may relate to defects in the source code. However, little is known about the developers' perception of these machine learning models and their explanations. To explore this issue, we focus on a survey with experienced developers to understand how they evaluate each quality attribute for the defect prediction. We chose the developers based on their contributions at GitHub, where they contributed to at least 10 repositories in the past 2 years. The results show that developers tend to evaluate code complexity as the most important quality attribute to avoid defects compared with the other target attributes such as source code size, coupling, and documentation. At the end, a thematic analysis reveals that developers evaluate testing the code as a relevant aspect not covered by the static software features. We conclude that, qualitatively, there exists a misalignment between developers' perceptions and the outputs of machine learning models. For instance, while machine learning models assign high importance to documentation, developers often overlook documentation and prioritize assessing the complexity of the code instead.</p>\u0000 </div>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 10","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Factors affecting architectural decision-making process and challenges in software projects: An industrial survey 影响软件项目中架构决策过程和挑战的因素:行业调查
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-18 DOI: 10.1002/smr.2703
Merve Özdeş Demir, Oumout Chouseinoglou, Ayça Kolukısa Tarhan
{"title":"Factors affecting architectural decision-making process and challenges in software projects: An industrial survey","authors":"Merve Özdeş Demir,&nbsp;Oumout Chouseinoglou,&nbsp;Ayça Kolukısa Tarhan","doi":"10.1002/smr.2703","DOIUrl":"https://doi.org/10.1002/smr.2703","url":null,"abstract":"<p>Software architecture plays a fundamental role in overcoming the challenges of the development process of large-scale and complex software systems. The software architecture of a system is the result of an extensive process in which several stakeholders negotiate issues and solutions, and as a result of this negotiation, a series of architectural decisions are made. This survey study aims to determine the experiences of the software industry experts with respect to architectural decision-making, the factors that are effective in decision-making, and the technical and social problems they encounter. An online questionnaire-based survey was conducted with 101 practitioners. The responses were analyzed qualitatively and quantitatively. Analysis of responses revealed that the majority of the participants prefer to document some or all of the architectural decisions taken and to store these documents in web-based collaboration software. Decisions are usually made by teams of two or three, and discussion-based approaches (brainstorming and consensus) are adopted. In the software architecture decision-making process, “major business impact” is the most challenging situation. Information sharing and keeping track of decisions and decision rationale are areas in need of improvement as identified by most participants. From the participants' feedback and their answers to open-ended questions, we concluded that the software architecture decision-making process has an important role in the industry. Our key findings are that decisions made in the architectural decision-making process are taken by teams and generally all decisions are documented. In projects where decisions are made by a single person, peer pressure is found to be significantly different from pressure in projects where decisions are made by the group. This is an indication that as the number of people in the decision-making process increases, the disagreements also increase.</p>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 10","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Hybrid mutation driven testing for natural language inference 自然语言推理的混合突变驱动测试
IF 1.7 4区 计算机科学
Journal of Software-Evolution and Process Pub Date : 2024-06-17 DOI: 10.1002/smr.2694
Linghan Meng, Yanhui Li, Lin Chen, Mingliang Ma, Yuming Zhou, Baowen Xu
{"title":"Hybrid mutation driven testing for natural language inference","authors":"Linghan Meng,&nbsp;Yanhui Li,&nbsp;Lin Chen,&nbsp;Mingliang Ma,&nbsp;Yuming Zhou,&nbsp;Baowen Xu","doi":"10.1002/smr.2694","DOIUrl":"https://doi.org/10.1002/smr.2694","url":null,"abstract":"<div>\u0000 \u0000 <p>Natural language inference (NLI) is a task to infer the relationship between the premise and hypothesis sentences, whose models have essential applications in the many natural language processing (NLP) fields, for example, machine reading comprehension and recognizing textual entailment. Due to the data-driven programming paradigm, bugs inevitably occur in NLI models during the application process, which calls for novel automatic testing techniques to deal with NLI testing challenges. The main difficulty in achieving automatic testing for NLI models is the oracle problem; that is, it may be too expensive to label NLI model inputs manually and hence be too challenging to verify the correctness of model outputs. To tackle the oracle problem, this study proposes a novel automatic testing method <b>hybrid mutation driven testing (HMT)</b>, which extends the mutation idea applied in other NLP domains successfully. Specifically, as there are two sets of sentences, that is, premise and hypothesis, to be mutated, we propose four mutation operators to achieve the hybrid mutation strategy, which mutate the premise and the hypothesis sentences <i>jointly</i> or <i>individually</i>. We assume that the mutation would not affect the outputs; that is, if the original and mutated outputs are inconsistent, inconsistency bugs could be detected without knowing the true labels. To evaluate our method HMT, we conduct experiments on two widely used datasets with two advanced models and generate more than 520,000 mutations by applying our mutation operators. Our experimental results show that (a) our method, HMT, can effectively generate mutated testing samples, (b) our method can effectively trigger the inconsistency bugs of the NLI models, and (c) all four mutation operators can independently trigger inconsistency bugs.</p>\u0000 </div>","PeriodicalId":48898,"journal":{"name":"Journal of Software-Evolution and Process","volume":"36 10","pages":""},"PeriodicalIF":1.7,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信