IEEE Transactions on Software Engineering最新文献

筛选
英文 中文
Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation 检索增强代码生成中知识库中毒的安全威胁探讨
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-04-30 DOI: 10.1109/TSE.2026.3689066
Bo Lin;Shangwen Wang;Liqian Chen;Xiaoguang Mao
{"title":"Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation","authors":"Bo Lin;Shangwen Wang;Liqian Chen;Xiaoguang Mao","doi":"10.1109/TSE.2026.3689066","DOIUrl":"https://doi.org/10.1109/TSE.2026.3689066","url":null,"abstract":"The integration of Large Language Models (LLMs) into software development has revolutionized the field, particularly through the use of Retrieval-Augmented Code Generation (RACG) systems that enhance code generation with information from external knowledge bases. However, the security implications of RACG systems, particularly the risks posed by vulnerable code examples in the knowledge base, remain largely unexplored. This risk is notably concerning given that public code repositories, which often serve as the sources for knowledge base collection in RACG systems, are usually accessible to anyone in the community. Malicious attackers can exploit this accessibility to inject vulnerable code into the knowledge base, making it toxic. Once these poisoned samples are retrieved and incorporated into the generated code, they can propagate security vulnerabilities into the final product. This paper presents the first comprehensive study on the security risks associated with RACG systems, focusing on how vulnerable code in the knowledge base compromises the security of generated code. We investigate the LLM-generated code security across different settings through extensive experiments using four major LLMs, two retrievers, and two poisoning scenarios. Our findings highlight the significant threat of knowledge base poisoning, where even a single poisoned code example can compromise up to 48% of the generated code. Our findings provide crucial insights into vulnerability introduction in RACG systems and offer practical mitigation recommendations, thereby helping improve the security of LLM-generated code in future works.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2250-2267"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148746922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Onboarding Software Professionals in a Hybrid World 在一个混合的世界里,软件专业人员的入职
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-03-19 DOI: 10.1109/TSE.2026.3694572
Klaas-Jan Stol;Viktoria Stray;Nils Brede Moe;Tor Sporsem
{"title":"Onboarding Software Professionals in a Hybrid World","authors":"Klaas-Jan Stol;Viktoria Stray;Nils Brede Moe;Tor Sporsem","doi":"10.1109/TSE.2026.3694572","DOIUrl":"10.1109/TSE.2026.3694572","url":null,"abstract":"Onboarding is an important activity to integrate new employees into an organization as soon as possible, in order to support them in becoming productive. Whereas onboarding has traditionally been done primarily onsite, it is now increasingly common for new software professionals to be onboarded either fully remotely or partially remotely in hybrid settings. We ask how software professionals develop a sense of belonging through the onboarding process. Using a conceptual framework from the general onboarding literature, we report on case studies of onboarding of software professionals in three different settings on the hybrid continuum, drawing on a dataset comprising interviews with 46 informants, Slack logs, observations, and company documents. In Case 1, we focus on the onboarding of consultant developers in two divisions: in one division, the consultants work internally with the consulting organization. In the other division, the consultants are placed with clients. In Case 2, we study the onboarding of software professionals in a hybrid setting, and in Case 3, we study the onboarding of a new team of remote developers based in Portugal, who joined a company in Norway. We extend the literature on onboarding of software professionals by shedding light on how software organizations conduct onboarding in different types of hybrid settings. We extend and refine the initial conceptual model, and formalize it through the development of a series of propositions. We then discuss a number of recommendations for software organizations who wish to onboard new software professionals in different settings on the hybrid continuum.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2383-2409"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11523726","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
CCMG: Enhancing Conventional Commit Message Generation With Hierarchical Context CCMG:通过分层上下文增强常规提交消息生成
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-03-12 DOI: 10.1109/TSE.2026.3692869
Wenke Li;Chengwei Liu;Yiran Zhang;Xuesen Lin;Suyuan Wang;Feng Wu;Cai Fu;Yang Liu
{"title":"CCMG: Enhancing Conventional Commit Message Generation With Hierarchical Context","authors":"Wenke Li;Chengwei Liu;Yiran Zhang;Xuesen Lin;Suyuan Wang;Feng Wu;Cai Fu;Yang Liu","doi":"10.1109/TSE.2026.3692869","DOIUrl":"10.1109/TSE.2026.3692869","url":null,"abstract":"Automated commit message generation, which aims at generating natural language description from code change, allows developers to focus more on project maintenance and management. To ensure the quality of commit messages, most projects constrain their style and adopt the conventional commit specification. Conventional commit message generation has been significantly benefited from recent progress in Large Language Models (LLMs). However, previous approaches typically rely on only one or two type of information for the generation, ignoring a wide range of context information. Moreover, they often extract the context in a coarse-grained manner, missing critical details. To address this limitation, We propose CCMG, a novel hierarchical context-augmented <bold> <u>C</u></b>onventional <bold> <u>C</u></b>ommit <bold> <u>M</u></b>essage <bold> <u>G</u></b>eneration framework, which incorporates project-agnostic and project-specific context. For project agnostic context, CCMG retrieves and refines the relevant commits to align conventional commit specification from large-scale corpus. For project-specific context, CCMG provides a wide range of software context information from the perspective of project, code, and style. Finally, CCMG designs two-stage prompt strategy to focus on conventional message inference and commit type adaptation. Compared with the state-of-the-art LLM-based approaches (i.e., OMG and OMEGA), experiment results show that CCMG achieves an average improvement of 31.45% based on human evaluation in commit message generation and improves accuracy by 21.00% and F1 score by 20.88% in commit type classification.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2303-2317"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Extracting Conditional Expressions as Local Variables: Frequency, Motivation, and Automated Recommendation 提取条件表达式作为局部变量:频率、动机和自动推荐
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-06-26 DOI: 10.1109/TSE.2026.3700827
Bridget Nyirongo;Yanjie Jiang;Yuxia Zhang;Hui Liu
{"title":"Extracting Conditional Expressions as Local Variables: Frequency, Motivation, and Automated Recommendation","authors":"Bridget Nyirongo;Yanjie Jiang;Yuxia Zhang;Hui Liu","doi":"10.1109/TSE.2026.3700827","DOIUrl":"10.1109/TSE.2026.3700827","url":null,"abstract":"Software refactoring is essential for improving code readability and maintainability while reducing technical debt. Applying the extract local variable refactoring to condition expressions can simplify complex logic, enhance clarity, and reduce control-flow complexity. However, despite these benefits, mainstream IDEs and recent tool enhancements lack targeted, context-aware recommendations for this specific transformation. Understanding the patterns and contextual factors driving such extractions is necessary to augment existing tools with heuristics that promote meaningful and maintainable changes. To address this gap, this paper presents the first empirical study of extract local variable refactorings applied to conditional expressions, analyzing 984 instances across ten popular open-source Java projects. Our analysis shows that approximately 10.1% of extract local variable refactorings target condition expressions, highlighting their non-trivial presence in real-world projects. Through a two-round questionnaire survey, we identify and prioritize seven key motivations for applying extract variable to condition expressions, offering insight into developer intent. We further identify and validate twenty recurring features of such expressions, informing a set of practical heuristics that provide structural signals for suggesting this refactoring. Building on these findings, we introduce <italic>ConExtractor</i>, a hybrid recommendation prototype combining heuristics with LLM evaluation. Across multiple LLM configurations, the hybrid approach achieves precision up to 96.7% and recall up to 88%, demonstrating its potential to provide selective and developer-aligned refactoring suggestions in practice.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2507-2527"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148342293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Toward Automated Discovery of Asymmetric Mempool DoS in Blockchains 区块链中不对称内存池DoS的自动发现
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-06-01 DOI: 10.1109/TSE.2026.3698791
Yibo Wang;Yuzhe Tang;Kai Li;Wanning Ding
{"title":"Toward Automated Discovery of Asymmetric Mempool DoS in Blockchains","authors":"Yibo Wang;Yuzhe Tang;Kai Li;Wanning Ding","doi":"10.1109/TSE.2026.3698791","DOIUrl":"10.1109/TSE.2026.3698791","url":null,"abstract":"In blockchains, mempool controls transaction flow before consensus, denial of whose service hurts the health and security of blockchain networks. This paper presents <small>mpfuzz</small>, the first mempool fuzzer to find asymmetric DoS bugs by exploring the space of symbolized mempool states and optimistically estimating the promisingness of an intermediate state in reaching bug oracles. Compared to the baseline blockchain fuzzers, <small>mpfuzz</small> achieves a <inline-formula><tex-math>$mathbf{gt 100times}$</tex-math></inline-formula> speedup in finding known DETER exploits. Running <small>mpfuzz</small> on major Ethereum clients leads to discovering new mempool vulnerabilities, which exhibit a wide variety of sophisticated patterns, including stealthy mempool eviction and mempool locking. Rule-based mitigation schemes are proposed against all newly discovered vulnerabilities.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2475-2487"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148288733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
UntrustVul: Automated Untrustworthy Alert Identification in Vulnerability Detection Models UNTRUSTVUL:漏洞检测模型中的自动不可信警报识别
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-03-19 DOI: 10.1109/TSE.2026.3693636
Lam Nguyen Tung;Xiaoning Du;Neelofar Neelofar;Aldeida Aleti
{"title":"UntrustVul: Automated Untrustworthy Alert Identification in Vulnerability Detection Models","authors":"Lam Nguyen Tung;Xiaoning Du;Neelofar Neelofar;Aldeida Aleti","doi":"10.1109/TSE.2026.3693636","DOIUrl":"10.1109/TSE.2026.3693636","url":null,"abstract":"Machine learning (ML) has shown promising results in detecting software vulnerabilities. However, ML detectors are not guaranteed to make predictions based on the right indicators. Studies have revealed that they can rely on <italic>irrelevant</i> code features, such as identifiers or function signatures, particularly those that commonly appear in vulnerable code, yet are not related to the actual vulnerabilities. As a result, the lines of code that the detectors depend on and flag as suspicious are not always genuinely vulnerable. Consequently, developers must manually review these suspicious lines, which is time-consuming and error-prone. If the suspicious lines are wrong, developers may be misled, spend unnecessary effort, or even reach incorrect patching strategies. This highlights the need for automated approaches to identify untrustworthy vulnerability predictions. In this paper, we introduce <sc>UntrustVul</small>, a new approach for identifying untrustworthy vulnerability predictions. Specifically, we focus on cases where a model highlights suspicious lines that would not appear in reliable predictions, i.e., lines that are inherently non-vulnerable and unrelated to any vulnerabilities. To achieve this, we leverage patterns of vulnerable lines observed in historical data. <sc>UntrustVul</small> automatically rules out as untrustworthy any predictions that highlight suspicious lines neither observed in history nor influential to those that have been observed. We refer to such lines as vulnerability-irrelevant. A line is deemed vulnerability-irrelevant if ① it does not match any known patterns of historical vulnerabilities, and ② all its successors in the data and control dependency graph are also vulnerability-irrelevant. Intuitively, a vulnerability-irrelevant line shows low similarity to known vulnerabilities and has no dependency paths to any lines outside the vulnerability-irrelevant category. Notably, these rules are designed to be conservative, as mislabeling a trustworthy prediction as untrustworthy is also undesired. We evaluate <sc>UntrustVul</small> on 115K vulnerability predictions made by four models across BigVul, MegaVul, SARD, and PrimeVul datasets, with ground-truth trustworthiness labeled based on the overlap between actual denoised vulnerable lines and model-annotated suspicious lines. <sc>UntrustVul</small> effectively detects untrustworthy predictions with AUC of 70%–88% and F1-score of 82%–94%, outperforming existing approaches by 6%–59% in AUC and 13%–92% in F1-score.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2335-2353"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291567","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
MalElves: Reinforcement Learning-Driven Adversarial Example Generation for Evading Cross-Platform ELF Malware Detection MalElves:用于逃避跨平台ELF恶意软件检测的强化学习驱动的对抗示例生成
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-03-14 DOI: 10.1109/TSE.2026.3693254
Zhangbo Long;Letian Sha;Yan Lin;Peijie Sun;Haiping Huang;Fu Xiao;Zhiquan Liu
{"title":"MalElves: Reinforcement Learning-Driven Adversarial Example Generation for Evading Cross-Platform ELF Malware Detection","authors":"Zhangbo Long;Letian Sha;Yan Lin;Peijie Sun;Haiping Huang;Fu Xiao;Zhiquan Liu","doi":"10.1109/TSE.2026.3693254","DOIUrl":"10.1109/TSE.2026.3693254","url":null,"abstract":"Adversarial Example (AE) generation is a key instrument for stress-testing and hardening malware detectors, yet most existing techniques target Portable Executable (PE) files and do not transfer cleanly to Executable and Linkable Format (ELF) binaries prevalent in Internet of Things (IoT) environments. We propose <monospace>MalElves</monospace>, a reinforcement learning-driven AE generation framework for cross-platform ELF malware. <monospace>MalElves</monospace> makes three core technical contributions. First, a code-data-aware manipulation framework unifies obfuscation and rewriting across ARM, x86, and x64 architectures while preserving functionality. Second, a sample-efficient state design reduces 2,350 raw ELF features to a compact 21-dimensional input. Third, a shaped multi-detector reward uses fully disclosed PPO settings for full reproducibility. We evaluate <monospace>MalElves</monospace> on 161,414 malware samples and 74,260 benign samples. We test against four static detectors and a behavioral-ensemble stress test. The method achieves average ASRs of 89.9%, 85.3%, 63.8%, 60.6%, and 24.7% across detectors. The overall average ASR reaches 64.8%. Each successful evasion requires 2.24 interaction rounds on average.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2318-2334"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Federated and Privacy-Preserving Cross-Project Defect Prediction With Heterogeneous Features 基于异构特征的联邦和隐私保护跨项目缺陷预测
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-03-11 DOI: 10.1109/TSE.2026.3692541
Meetesh Nevendra;Rahul Shrivastava
{"title":"Federated and Privacy-Preserving Cross-Project Defect Prediction With Heterogeneous Features","authors":"Meetesh Nevendra;Rahul Shrivastava","doi":"10.1109/TSE.2026.3692541","DOIUrl":"10.1109/TSE.2026.3692541","url":null,"abstract":"Organizations are often reluctant to share defect-labeled source code because of privacy and intellectual property concerns, yet effective cross-project defect prediction (CPDP) depends on learning from diverse projects. This paper introduces FedCPDP, a federated and privacy-preserving framework that enables collaborative defect prediction without exchanging raw project data and without requiring a common feature schema. The approach employs feature hashing to unify heterogeneous metric spaces, integrates federated proximal optimization with representation alignment to mitigate distribution shift, and incorporates differential privacy mechanisms to safeguard sensitive updates. To further enhance robustness, FedCPDP explores adaptive server aggregation strategies (attention and graph-based) and seed ensembling, while addressing class imbalance through focal loss and threshold optimization. The framework is evaluated on thirty benchmark datasets across five CPDP families under leave-one-project-out validation, using both threshold-free and effort-aware metrics. Results show that FedCPDP improves F1 by up to 10-15% over FedAvg and 6-8% over FedDPI, while achieving 8-10% higher <inline-formula><tex-math>$boldsymbol{P_{opt}}$</tex-math></inline-formula> under effort-aware evaluation. These gains are statistically significant, confirming that FedCPDP consistently outperforms state-of-the-art methods while uniquely supporting heterogeneous features and offering explicit privacy guarantees. These findings establish FedCPDP as a practical step toward enabling secure and deployable defect prediction in multi-organizational settings.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2268-2281"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
UntCC: Untangling Composite Commits Using Structural and Semantic Information UNTCC:使用结构和语义信息解开复合提交的缠结
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-03-11 DOI: 10.1109/TSE.2026.3692448
Yuzhe Jin;Lanxin Yang;He Zhang;Gongyuan Li;Jun Lyu;Bohan Liu;Xin Zhou;Hongyu Kuang;Liming Dong
{"title":"UntCC: Untangling Composite Commits Using Structural and Semantic Information","authors":"Yuzhe Jin;Lanxin Yang;He Zhang;Gongyuan Li;Jun Lyu;Bohan Liu;Xin Zhou;Hongyu Kuang;Liming Dong","doi":"10.1109/TSE.2026.3692448","DOIUrl":"10.1109/TSE.2026.3692448","url":null,"abstract":"Small and focused commits are highly valued in modern software development. However, developers sometimes submit a commit with more than one concern, represented by several lines of code changes for a specific purpose, <i>e.g</i>., adding new features or fixing bugs. Such composite commits confuse developers during code reviews as well as other software activities, resulting in various issues. Existing studies predominantly leverage code structure to untangle composite commits, but without considering code semantics that have been demonstrated to be important in many related studies. In this article, we propose <small>UntCC</small>, a new approach that uses structural and semantic information for <u>U<small>nt</small> </u>angling <u>C</u>omposite <u>C</u>ommits. To achieve structural information, we propose the code change graph, a fine-grained, text-attributed graph representation of a commit, incorporating before-change and after-change code dependencies; and <small>UntCC</small> employs the graph autoencoder to learn its structural representation. To achieve semantic information, <small>UntCC</small> leverages a large language model (Llama-3.2-3B) to learn joint embeddings of the raw commit and its aligned graph representation, which guide the division of different concerns within a composite commit. The experimental evaluation using 27,853 composite commits from 9 C# and 10 Java projects shows that in terms of <inline-formula><tex-math>$mathbf{Accuracy}^{mathbf{a}}$</tex-math></inline-formula>/<inline-formula><tex-math>$mathbf{Accuracy}^{mathbf{c}}$</tex-math></inline-formula>, <small>UntCC</small> achieves 94%/74% in C# and 77%/54% in Java, outperforming state-of-the-art approaches by 2%–623%/32%–573% in C# and 22%–285%/35%–286% in Java. The results indicate that <small>UntCC</small> can effectively untangle composite commits.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2282-2302"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Framework for Evaluating GenAI Adoption and Use in Software Engineering 软件工程中GenAI采用和使用的评估框架
IF 6 1区 计算机科学
IEEE Transactions on Software Engineering Pub Date : 2026-08-01 Epub Date: 2026-04-29 DOI: 10.1109/TSE.2026.3688745
Liang Yu;Emil Alégroth;Panagiota Chatzipetrou;Tony Gorschek
{"title":"A Framework for Evaluating GenAI Adoption and Use in Software Engineering","authors":"Liang Yu;Emil Alégroth;Panagiota Chatzipetrou;Tony Gorschek","doi":"10.1109/TSE.2026.3688745","DOIUrl":"https://doi.org/10.1109/TSE.2026.3688745","url":null,"abstract":"Generative Artificial Intelligence (GenAI) is increasingly integrated into software products to enable new features and user capabilities, from early exploration to operational deployment. GenAI adoption as a component within a software system introduces quality risks because GenAI outputs are probabilistic, prompt-sensitive, and may drift after release. Organizations, therefore, need to decide what to evaluate, when to evaluate, and who owns quality evaluation activities across software design, development, and operations. ISO/IEC 25059 standard distinguishes between software product quality (e.g., usability) and quality-in-use (e.g., satisfaction) for AI-enabled software, yet it provides limited operational guidance for these evaluation activities. We therefore investigate how industrial software teams adopt and use GenAI models in the software systems they build and operate, and how they evaluate system qualities when deciding to adopt GenAI during development and after deployment. We do not benchmark the underlying GenAI model itself. In this study, we conducted 19 semi-structured interviews in two software development companies. We triangulated the interviews with archival data (15 internal documents and 184 internal wiki/web pages) to capture GenAI adoption steps, quality concerns, evaluation practices, and role responsibilities. Our findings describe a three-phase adoption process – Ideation, Development, and Operation – highlighting where quality evaluations occur, which criteria are used, and how evaluation responsibilities are distributed. Based on observed practices and using ISO/IEC 25059 as an organizing lens, we synthesize a process-oriented quality evaluation framework. This framework maps metrics to explicit gatekeeping, validation, and monitoring checkpoints, bridging abstract ISO quality characteristics with engineering workflows. We applied the framework in a GenAI-enabled software product (SE4AI) use case and reported how it supported structured evaluation activities. We also observed that quality evaluations span legal, security, development, QA, and operations, but ownership is fragmented across phases. We therefore propose a GenAI Quality Lead responsibility (often assignable to an existing senior role) to coordinate criteria, evidence, and traceability across quality evaluation activities. The results contribute to Software Engineering for AI (SE4AI) by clarifying how teams can measure qualities when building software that adopts and uses GenAI.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2233-2249"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11499432","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148749661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书