IEEE Transactions on Software Engineering最新文献

{"title":"Agentic Program Repair From Test Failures at Scale: A Neuro-Symbolic Approach With Static Analysis and Test Execution Feedback","authors":"Chandra Maddila;Adam Tait;Claire Chang;Daniel Cheng;Nauman Ahmad;Vijayaraghavan Murali;Marshall Roch;Arnaud Avondet;Aaron Meltzer;Victor Montalvao;Michael Hopko;Chris Waterson;Parth Thakkar;Renuka Fernandez;Kristian Kristensen;Sivan Barzily;Sherry Chen;Rui Abreu;Nachiappan Nagappan;Payam Shodjai;Killian Murphy;James Everingham;Aparna Ramani;Peter C. Rigby","doi":"10.1109/TSE.2026.3696849","DOIUrl":"10.1109/TSE.2026.3696849","url":null,"abstract":"<bold>Aim.</b> With the advent of LLMs, agentic program repair has become viable in large organizations with large codebases. While APR research frequently motivates its techniques around helping developers, Winter et al. <xref>[1]</xref> report that studies with substantial industrial participants are rare (2%). In this work, we develop an Engineering Agent that fixes source code from test failures at scale across diverse software offerings internally. <p><b>Method.</b> Using Llama as the base model, we employ a ReAct harness to build an agent that starts from a test failure triaged by a rule-based test failure bot. The agent operates in an agentic harness with 15 actions (e.g., reading files, searching code, running tests, and generating patches), and it iteratively refines solutions using feedback from static analysis and test execution. After validations pass, we apply an LLM-as-a-Judge to filter low-quality patches before notifying a human reviewer, who can accept and land the change in our monorepo. Because we can progressively roll out the system with layered validation and human review, we use offline benchmarks primarily as readiness gates and complement them with production evaluation based on review and landing outcomes.</p> <p><b>Benchmark Findings.</b> We curated offline benchmarks for patch generation, the agent loop, and the LLM-as-a-Judge. We found that a specialized 70B model, internally fine-tuned for patch generation in a search-and-replace format, is highly competitive with a much larger vanilla Llama-405B. We also found that patch format strongly impacts performance, with search-and-replace outperforming unified diff. In ablation studies, symbolic feedback from static analysis and test execution improves the ReAct agent. The balanced configuration achieves a benchmark solve rate of 42.3% using an average 11.8 feedback iterations.</p> <p><b>Production Findings.</b> Over a three month period, 80% of the generated fixes were reviewed, of which 31.5% were landed (25.5% of the total number of generated fixes).</p> <p><b>Engineer Feedback and Adoption.</b> We used open coding to extract qualitative themes from engineer feedback, including quick approvals, gratitude, and surprise, and we used negative feedback (e.g., test flakiness) to refine the production system. We also analyzed reviewed-but-not-merged diffs to understand why validated patches are rejected in practice, including workflow timing, intent mismatches (revert vs. fix-forward), and broader patch acceptability concerns.</p>","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2446-2462"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem","authors":"Hao Song;Yiming Shen;Wenxuan Luo;Leixin Guo;Ting Chen;Jiashui Wang;Beibei Li;Xiaosong Zhang;Jiachi Chen","doi":"10.1109/TSE.2026.3694876","DOIUrl":"10.1109/TSE.2026.3694876","url":null,"abstract":"The Model Context Protocol (MCP) is an emerging standard designed to enable seamless interaction between Large Language Model (LLM) applications and external tools or resources. Within a short period, thousands of MCP services have been developed and deployed. However, the client-server integration architecture inherent in MCP may expand the attack surface against LLM Agent systems, introducing new vulnerabilities allowing attackers to exploit by malicious MCP servers. In this paper, we present the <b><i>first end-to-end empirical evaluation</i></b> of attack vectors targeting the MCP ecosystem. We provide the formal definitions, affected path, and exploited path based on the workflow for four malicious server attacks, i.e., <i>Tool Poisoning Attacks</i>, <i>Puppet Attacks</i>, <i>Rug Pull Attacks</i>, and <i>Exploitation via Malicious External Resources</i>. To evaluate their feasibility, we conduct experiments following the typical steps of launching an attack through malicious MCP servers: upload <inline-formula><tex-math>$rightarrow$</tex-math></inline-formula> download <inline-formula><tex-math>$rightarrow$</tex-math></inline-formula> attack. Specifically, we first construct malicious MCP servers and successfully upload them to three widely used MCP aggregation platforms. The results indicate that current audit mechanisms are insufficient to identify and prevent these threats. Next, through a user study and interview with 20 participants, we demonstrate that users struggle to identify malicious MCP servers and often unknowingly install them from aggregator platforms. Finally, we empirically demonstrate that these attacks can trigger harmful actions within the user’s local environment, such as accessing private files or controlling devices to transfer digital assets. Additionally, based on interview results, we discuss four key challenges faced by the current MCP security ecosystem. These findings underscore the urgent need for robust security mechanisms to defend against malicious MCP servers and ensure the safe deployment of increasingly autonomous LLM agents.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2410-2426"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Large Language Model Assisted Hybrid Fuzzing","authors":"Ruijie Meng;Gregory J. Duck;Abhik Roychoudhury","doi":"10.1109/TSE.2026.3694408","DOIUrl":"https://doi.org/10.1109/TSE.2026.3694408","url":null,"abstract":"Greybox fuzzing is one of the most popular methods for detecting software vulnerabilities, which conducts a biased random search within the program input space. To enhance its effectiveness in achieving deep coverage of program behaviors, greybox fuzzing is often combined with concolic execution, which performs a path-sensitive search over the domain of program inputs. In hybrid fuzzing, conventional greybox fuzzing is followed by concolic execution in an iterative loop, where reachability roadblocks encountered by greybox fuzzing are tackled by concolic execution. However, such hybrid fuzzing still suffers from difficulties conventionally faced by concolic execution, such as the need for environment modeling and system call support. In this work, we explore the potential of developing “smart” concolic execution empowered by Large Language Models (LLMs), leveraging their knowledge of code semantics during constraint computing and solving. When coverage-based greybox fuzzing reaches a roadblock in terms of reaching certain branches, we conduct a slicing on the execution trace and suggest modifications of the input to reach the relevant branches. The LLM is used as a solver to generate the modified input to reach the desired branches. Compared with state-of-the-art hybrid fuzzers <sc>CoFuzz</small>, <sc>Intriguer</small>, and QSYM, our LLM-based hybrid fuzzer <sc>HyLLfuzz</small> (pronounced “<italic>hill fuzz</i>”) covers 31.43%, 44.56%, and 59.48% more code branches, respectively. Furthermore, the LLM-based concolic execution in <sc>HyLLfuzz</small> takes a time that is 3–19 times faster than the concolic execution running in existing hybrid fuzzing tools. In extensively tested real-world subjects, <sc>HyLLfuzz</small> exposed seven previously unknown bugs. This experience shows that LLMs can be effectively inserted into the iterative loop of hybrid fuzzers to efficiently expose more program behaviors.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2367-2382"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148748940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Tackling Expressive Feature-Modeling Constructs With Pseudo-Boolean d-DNNF Compilation","authors":"Chico Sundermann;Stefan Vill;Elias Kuiter;Sebastian Krieter;Thomas Thüm;Matthias Tichy","doi":"10.1109/TSE.2026.3696093","DOIUrl":"10.1109/TSE.2026.3696093","url":null,"abstract":"Configurable systems typically consist of reusable assets that have dependencies between each other. To specify such dependencies, feature models are commonly used. As feature models in practice are often complex, automated reasoning is typically employed to analyze the dependencies. Here, the de facto standard is translating the feature model to conjunctive normal form (CNF) to enable employing off-the-shelf tools, such as SAT or <monospace>#</monospace>SAT solvers. However, modern feature-modeling dialects often contain constructs, such as cardinality constraints, that are ill-suited for conversion to CNF. This mismatch between the input of reasoning engines and the available feature-modeling dialects limits the applicability of the more expressive constructs. In this work, we shorten this gap between expressive constructs and scalable automated reasoning. Our contribution is twofold: First, we provide a pseudo-Boolean encoding for feature models, which facilitates smaller representations of commonly employed constructs compared to Boolean encoding. Second, we propose a novel method to compile pseudo-Boolean formulas to Boolean d-DNNFs. With the compiled d-DNNFs, we can resort to a plethora of efficient analyses already used in feature modeling. Our empirical evaluation shows that our proposal substantially outperforms the state-of-the-art based on CNF inputs for expressive constructs. For every considered dataset representing different feature models and feature-modeling constructs, the feature models can be significantly faster translated to pseudo-Boolean than to CNF. Overall, deriving d-DNNFs from a feature model with the targeted expressive constraints can be substantially accelerated using our pseudo-Boolean approach. For instance, the Boolean approach only scales for group cardinalities with up-to 13 features while pseudo-Boolean d-DNNF compilation can compile cardinalities with thousands of features. Furthermore, our approach is competitive on feature models with only basic constructs.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2427-2445"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11534432","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation","authors":"Bo Lin;Shangwen Wang;Liqian Chen;Xiaoguang Mao","doi":"10.1109/TSE.2026.3689066","DOIUrl":"https://doi.org/10.1109/TSE.2026.3689066","url":null,"abstract":"The integration of Large Language Models (LLMs) into software development has revolutionized the field, particularly through the use of Retrieval-Augmented Code Generation (RACG) systems that enhance code generation with information from external knowledge bases. However, the security implications of RACG systems, particularly the risks posed by vulnerable code examples in the knowledge base, remain largely unexplored. This risk is notably concerning given that public code repositories, which often serve as the sources for knowledge base collection in RACG systems, are usually accessible to anyone in the community. Malicious attackers can exploit this accessibility to inject vulnerable code into the knowledge base, making it toxic. Once these poisoned samples are retrieved and incorporated into the generated code, they can propagate security vulnerabilities into the final product. This paper presents the first comprehensive study on the security risks associated with RACG systems, focusing on how vulnerable code in the knowledge base compromises the security of generated code. We investigate the LLM-generated code security across different settings through extensive experiments using four major LLMs, two retrievers, and two poisoning scenarios. Our findings highlight the significant threat of knowledge base poisoning, where even a single poisoned code example can compromise up to 48% of the generated code. Our findings provide crucial insights into vulnerability introduction in RACG systems and offer practical mitigation recommendations, thereby helping improve the security of LLM-generated code in future works.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2250-2267"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148746922","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Onboarding Software Professionals in a Hybrid World","authors":"Klaas-Jan Stol;Viktoria Stray;Nils Brede Moe;Tor Sporsem","doi":"10.1109/TSE.2026.3694572","DOIUrl":"10.1109/TSE.2026.3694572","url":null,"abstract":"Onboarding is an important activity to integrate new employees into an organization as soon as possible, in order to support them in becoming productive. Whereas onboarding has traditionally been done primarily onsite, it is now increasingly common for new software professionals to be onboarded either fully remotely or partially remotely in hybrid settings. We ask how software professionals develop a sense of belonging through the onboarding process. Using a conceptual framework from the general onboarding literature, we report on case studies of onboarding of software professionals in three different settings on the hybrid continuum, drawing on a dataset comprising interviews with 46 informants, Slack logs, observations, and company documents. In Case 1, we focus on the onboarding of consultant developers in two divisions: in one division, the consultants work internally with the consulting organization. In the other division, the consultants are placed with clients. In Case 2, we study the onboarding of software professionals in a hybrid setting, and in Case 3, we study the onboarding of a new team of remote developers based in Portugal, who joined a company in Norway. We extend the literature on onboarding of software professionals by shedding light on how software organizations conduct onboarding in different types of hybrid settings. We extend and refine the initial conceptual model, and formalize it through the development of a series of propositions. We then discuss a number of recommendations for software organizations who wish to onboard new software professionals in different settings on the hybrid continuum.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2383-2409"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11523726","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CCMG: Enhancing Conventional Commit Message Generation With Hierarchical Context","authors":"Wenke Li;Chengwei Liu;Yiran Zhang;Xuesen Lin;Suyuan Wang;Feng Wu;Cai Fu;Yang Liu","doi":"10.1109/TSE.2026.3692869","DOIUrl":"10.1109/TSE.2026.3692869","url":null,"abstract":"Automated commit message generation, which aims at generating natural language description from code change, allows developers to focus more on project maintenance and management. To ensure the quality of commit messages, most projects constrain their style and adopt the conventional commit specification. Conventional commit message generation has been significantly benefited from recent progress in Large Language Models (LLMs). However, previous approaches typically rely on only one or two type of information for the generation, ignoring a wide range of context information. Moreover, they often extract the context in a coarse-grained manner, missing critical details. To address this limitation, We propose CCMG, a novel hierarchical context-augmented <bold> <u>C</u></b>onventional <bold> <u>C</u></b>ommit <bold> <u>M</u></b>essage <bold> <u>G</u></b>eneration framework, which incorporates project-agnostic and project-specific context. For project agnostic context, CCMG retrieves and refines the relevant commits to align conventional commit specification from large-scale corpus. For project-specific context, CCMG provides a wide range of software context information from the perspective of project, code, and style. Finally, CCMG designs two-stage prompt strategy to focus on conventional message inference and commit type adaptation. Compared with the state-of-the-art LLM-based approaches (i.e., OMG and OMEGA), experiment results show that CCMG achieves an average improvement of 31.45% based on human evaluation in commit message generation and improves accuracy by 21.00% and F1 score by 20.88% in commit type classification.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2303-2317"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Extracting Conditional Expressions as Local Variables: Frequency, Motivation, and Automated Recommendation","authors":"Bridget Nyirongo;Yanjie Jiang;Yuxia Zhang;Hui Liu","doi":"10.1109/TSE.2026.3700827","DOIUrl":"10.1109/TSE.2026.3700827","url":null,"abstract":"Software refactoring is essential for improving code readability and maintainability while reducing technical debt. Applying the extract local variable refactoring to condition expressions can simplify complex logic, enhance clarity, and reduce control-flow complexity. However, despite these benefits, mainstream IDEs and recent tool enhancements lack targeted, context-aware recommendations for this specific transformation. Understanding the patterns and contextual factors driving such extractions is necessary to augment existing tools with heuristics that promote meaningful and maintainable changes. To address this gap, this paper presents the first empirical study of extract local variable refactorings applied to conditional expressions, analyzing 984 instances across ten popular open-source Java projects. Our analysis shows that approximately 10.1% of extract local variable refactorings target condition expressions, highlighting their non-trivial presence in real-world projects. Through a two-round questionnaire survey, we identify and prioritize seven key motivations for applying extract variable to condition expressions, offering insight into developer intent. We further identify and validate twenty recurring features of such expressions, informing a set of practical heuristics that provide structural signals for suggesting this refactoring. Building on these findings, we introduce <italic>ConExtractor</i>, a hybrid recommendation prototype combining heuristics with LLM evaluation. Across multiple LLM configurations, the hybrid approach achieves precision up to 96.7% and recall up to 88%, demonstrating its potential to provide selective and developer-aligned refactoring suggestions in practice.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2507-2527"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148342293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward Automated Discovery of Asymmetric Mempool DoS in Blockchains","authors":"Yibo Wang;Yuzhe Tang;Kai Li;Wanning Ding","doi":"10.1109/TSE.2026.3698791","DOIUrl":"10.1109/TSE.2026.3698791","url":null,"abstract":"In blockchains, mempool controls transaction flow before consensus, denial of whose service hurts the health and security of blockchain networks. This paper presents <small>mpfuzz</small>, the first mempool fuzzer to find asymmetric DoS bugs by exploring the space of symbolized mempool states and optimistically estimating the promisingness of an intermediate state in reaching bug oracles. Compared to the baseline blockchain fuzzers, <small>mpfuzz</small> achieves a <inline-formula><tex-math>$mathbf{gt 100times}$</tex-math></inline-formula> speedup in finding known DETER exploits. Running <small>mpfuzz</small> on major Ethereum clients leads to discovering new mempool vulnerabilities, which exhibit a wide variety of sophisticated patterns, including stealthy mempool eviction and mempool locking. Rule-based mitigation schemes are proposed against all newly discovered vulnerabilities.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2475-2487"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148288733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"UntrustVul: Automated Untrustworthy Alert Identification in Vulnerability Detection Models","authors":"Lam Nguyen Tung;Xiaoning Du;Neelofar Neelofar;Aldeida Aleti","doi":"10.1109/TSE.2026.3693636","DOIUrl":"10.1109/TSE.2026.3693636","url":null,"abstract":"Machine learning (ML) has shown promising results in detecting software vulnerabilities. However, ML detectors are not guaranteed to make predictions based on the right indicators. Studies have revealed that they can rely on <italic>irrelevant</i> code features, such as identifiers or function signatures, particularly those that commonly appear in vulnerable code, yet are not related to the actual vulnerabilities. As a result, the lines of code that the detectors depend on and flag as suspicious are not always genuinely vulnerable. Consequently, developers must manually review these suspicious lines, which is time-consuming and error-prone. If the suspicious lines are wrong, developers may be misled, spend unnecessary effort, or even reach incorrect patching strategies. This highlights the need for automated approaches to identify untrustworthy vulnerability predictions. In this paper, we introduce <sc>UntrustVul</small>, a new approach for identifying untrustworthy vulnerability predictions. Specifically, we focus on cases where a model highlights suspicious lines that would not appear in reliable predictions, i.e., lines that are inherently non-vulnerable and unrelated to any vulnerabilities. To achieve this, we leverage patterns of vulnerable lines observed in historical data. <sc>UntrustVul</small> automatically rules out as untrustworthy any predictions that highlight suspicious lines neither observed in history nor influential to those that have been observed. We refer to such lines as vulnerability-irrelevant. A line is deemed vulnerability-irrelevant if ① it does not match any known patterns of historical vulnerabilities, and ② all its successors in the data and control dependency graph are also vulnerability-irrelevant. Intuitively, a vulnerability-irrelevant line shows low similarity to known vulnerabilities and has no dependency paths to any lines outside the vulnerability-irrelevant category. Notably, these rules are designed to be conservative, as mislabeling a trustworthy prediction as untrustworthy is also undesired. We evaluate <sc>UntrustVul</small> on 115K vulnerability predictions made by four models across BigVul, MegaVul, SARD, and PrimeVul datasets, with ground-truth trustworthiness labeled based on the overlap between actual denoised vulnerable lines and model-annotated suspicious lines. <sc>UntrustVul</small> effectively detects untrustworthy predictions with AUC of 70%–88% and F1-score of 82%–94%, outperforming existing approaches by 6%–59% in AUC and 13%–92% in F1-score.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 8","pages":"2335-2353"},"PeriodicalIF":6.0,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148291567","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}