IEEE Transactions on Software Engineering最新文献

{"title":"FlakyAssessor: Static Assessment of Patch Correctness for Flaky Tests","authors":"Zhenyu Wu, Zheyuan Li, Yan Lei, Deheng Yang, Junyi Li, Huan Xie, Chunyan Liu","doi":"10.1109/tse.2026.3732261","DOIUrl":"https://doi.org/10.1109/tse.2026.3732261","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"16 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2026-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148909191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Agent System Operations: Categorization, Challenges, and Future Directions","authors":"Zexin Wang, Changhua Pei, Yuanhao Liu, Jingjing Li, Yintong Huo, Quan Zhou, Haotian Si, Hang Cui, Zihan Liu, Jianhui Li, Gaogang Xie, Fei Sun, Dan Pei, David Lo","doi":"10.1109/tse.2026.3732207","DOIUrl":"https://doi.org/10.1109/tse.2026.3732207","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"7 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2026-09-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148909190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"How Developers Respond to Insecure AI Code Suggestions under Poisoning Attacks: An Empirical Study of Interface-Level Countermeasures","authors":"Sanghak Oh, Eun Jung, Sangjun Chae, Hyoungshick Kim","doi":"10.1109/tse.2026.3731966","DOIUrl":"https://doi.org/10.1109/tse.2026.3731966","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"34 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2026-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148902032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Detecting LLM Fact-conflicting Hallucinations Enhanced by Temporal-logic-based Reasoning","authors":"Ningke Li, Yahui Song, Yuekang Li, Ling Shi, Yi Liu, Haoyu Wang, Kailong Wang","doi":"10.1109/tse.2026.3731191","DOIUrl":"https://doi.org/10.1109/tse.2026.3731191","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"14 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2026-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148902033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"LongTest: Test Prioritization for Long Text Files","authors":"Xueqi Dang;Yinghua Li;Wendkûuni C. Ouédraogo;Maxime Cordy;Mike Papadakis;Jacques Klein;Tegawendé F. Bissyandé;Yves Le Traon","doi":"10.1109/TSE.2026.3703637","DOIUrl":"10.1109/TSE.2026.3703637","url":null,"abstract":"Text classification is a fundamental task in natural language processing (NLP) that involves assigning predefined labels to text files. While existing research has primarily focused on short texts (e.g., social media posts), the classification of long texts, such as legal documents and medical records, has gained increasing attention due to the rising demand for analyzing complex and information-rich documents. However, testing long text classification systems poses unique challenges due to the high manual labeling costs associated with annotating test inputs (long text files). Test input prioritization has emerged as a promising strategy to mitigate this challenge, which prioritizes potentially misclassified tests during the early stages of the testing process. By labeling these inputs earlier, it accelerates the debugging process and thus improves the efficiency of DNN testing. However, applying existing methods to prioritize long text files is constrained by several factors: 1) Confidence-based methods fail to leverage the rich semantic content of the test inputs (long text files); 2) mutation-based techniques are specifically designed for short texts, introducing small mutations that have negligible impacts on long text files; and 3) coverage-based methods are computationally expensive and less effective. To overcome these limitations, we propose LongTest, a novel test input prioritization approach specifically designed for long text files. LongTest introduces: <b>1) an embedding generation mechanism</b> specifically designed to enhance the capture of information from the entire long text file, and <b>2) contrastive learning</b> that enables more effective differentiation between misclassified and correctly classified samples, which optimizes the test prioritization process. The results demonstrate that LongTest outperforms all the compared test prioritization approaches. Specifically, compared with the existing test prioritization methods (i.e., DeepGini, VanillaSM, PCS, Entropy, and DATIS), LongTest achieves an average improvement of 9.61%<inline-formula><tex-math>$sim$</tex-math></inline-formula>16.31%.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2789-2820"},"PeriodicalIF":6.0,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148288631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MORTAR: Multi-Turn Metamorphic Testing for LLM-Based Dialogue Systems","authors":"Guoxiang Guo;Aldeida Aleti;Neelofar Neelofar;Chakkrit Tantithamthavorn;Yuanyuan Qi;Tsong Yueh Chen","doi":"10.1109/TSE.2026.3701230","DOIUrl":"10.1109/TSE.2026.3701230","url":null,"abstract":"With the widespread application of LLM-based dialogue systems in daily life, quality assurance has become more important than ever. Recent research has successfully introduced methods to identify unexpected behaviour in single-turn testing scenarios. However, multi-turn interaction is the common real-world usage of dialogue systems, yet testing methods for such interactions remain underexplored. This is largely due to the oracle problem in multi-turn testing, which continues to pose a significant challenge for dialogue system developers and researchers. In this paper, we propose MORTAR, a metamorphic multi-turn dialogue testing approach, which mitigates the test oracle problem in testing LLM-based dialogue systems. MORTAR formalises multi-turn testing for dialogue systems, and automates the generation of question-answer dialogue test cases with multiple dialogue-level perturbations and metamorphic relations (MRs). The automated MR matching mechanism provides MORTAR more flexibility and efficiency in testing. When applied to eight LLM-based dialogue systems as test objects, MORTAR reveals 51% more bugs per test case than the best-performing single-turn metamorphic testing baseline, with 24.6% higher precision and over 42% of detected bugs being unique.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2620-2637"},"PeriodicalIF":6.0,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148288675","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An Empirical Investigation on Source Code Comprehension Performance Between Naming and Structural Issues With and Without Generative AI Assistance","authors":"Shotaro Inoue;Shuji Morisaki;Takuto Wada;Yuki Hattori","doi":"10.1109/TSE.2026.3702933","DOIUrl":"https://doi.org/10.1109/TSE.2026.3702933","url":null,"abstract":"Context: Software developers frequently comprehend source code with code smells that reduce readability. A previous study conducted without generative AI assistance reported that source code with naming issues required greater cognitive loads than that with structural issues. Objective: This study investigated whether accuracy and time differed when comprehending source code with naming or structural issues with and without generative AI assistance. Method: We conducted two experiments with 106 software development practitioners. In the first experiment, the subjects comprehended the source code with either naming or structural issues without generative AI assistance and answered nine questions about the source code. In the second experiment, the subjects performed the same task but with generative AI assistance. Additionally, we performed propensity score matching and compared the subjects in both experiments. Results: In the first experiment, unpaired <italic>t</i>-test results did not show a statistically significant difference in the percentage of correct answers between naming (mean 70.5%) and structural (mean 79.8%) issues after applying Holm’s correction (adjusted <italic>p</i> = 0.0652, <italic>p</i> = 0.0326) and no significant difference in the comprehension time (adjusted <italic>p</i> = 0.329, <italic>p</i> = 0.329). In the second experiment, no significant differences in the percentage of correct answers or comprehension time between naming and structural issues were observed. In the propensity score matching comparison, 25 pairs were matched. The results of the unpaired <italic>t</i>-test did not show a statistically significant difference in comprehension time between those without generative AI assistance (56.2 min) and those with assistance (45.1 min) when comprehending source code with naming issues (adjusted <italic>p</i> = 0.0622, <italic>p</i> = 0.0311). There was a significant difference in the percentage of correct answers between those without generative AI assistance (78.2%) and those with assistance (66.7%) when comprehending source code with structural issues (adjusted <italic>p</i> = 0.0354, <italic>p</i> = 0.0177). Conclusion: This study indicates that the effectiveness of generative AI assistance can be influenced by differences in the types of code smells in the source code and suggests why the effectiveness of generative AI assistance in source code comprehension has varied across previous studies.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2704-2720"},"PeriodicalIF":6.0,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11561013","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148985844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CCISolver: End-to-End Detection and Repair of Method-Level Code-Comment Inconsistency","authors":"Renyi Zhong;Yintong Huo;Wenwei Gu;Jinxi Kuang;Zhihan Jiang;Guangba Yu;Yichen Li;David Lo;Michael R. Lyu","doi":"10.1109/TSE.2026.3689089","DOIUrl":"https://doi.org/10.1109/TSE.2026.3689089","url":null,"abstract":"Comments within code serve as a crucial foundation for software documentation, facilitating developers to communicate and understand the code effectively. However, code-comment inconsistency (CCI) can negatively affect software development, testing, and maintenance. Recent efforts to mitigate this issue have emerged, but existing studies often suffer from inaccurate datasets and inadequate solutions, weakening their practical effectiveness. In this study, we first conduct a quantitative analysis of existing datasets, revealing that a substantial portion of the sampled data is mislabeled. To address these data limitations, we introduce <sc>CCIBench</small>, a refined dataset comprising high-quality data, to support the training and evaluation of method-level CCI methods. Furthermore, we present an innovative end-to-end LLM-based framework, <sc>CCISolver</small>, designed to improve code quality by identifying and rectifying CCIs. Comprehensive evaluations demonstrate <sc>CCISolver</small>’s superior performance. For detection, it establishes a new state-of-the-art with an F1-score of 89.54%. In the fixing task, it achieves a remarkable 18.84% relative improvement in GLEU score over the strongest baseline. This superiority is confirmed by human evaluation, where <sc>CCISolver</small>’s fixing success rate of 0.6533 significantly surpasses existing methods. Critically, <sc>CCISolver</small>’s architecture is approximately 36% faster during inference than the baseline in practical end-to-end settings. This efficiency underscores its scalability and real-world applicability.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2546-2565"},"PeriodicalIF":6.0,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148985388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SmarTracker: Leveraging LLMs to Automate User Behavior Tracking in Mobile Apps","authors":"Shuohan Wu;Hao Zhou;Kaifa Zhao;Le Yu;Shudi Bao;Xiapu Luo","doi":"10.1109/TSE.2026.3701153","DOIUrl":"10.1109/TSE.2026.3701153","url":null,"abstract":"The explosive growth of the mobile application market makes it a significant challenge for developers to increase their user retention. To deliver personalized experiences that attract and engage users, developers are placing greater emphasis on understanding user intentions and building detailed user profiles. A common practice is to insert tracking code during app development to log in-app user behaviors, such as button clicks and page visits, which are used in downstream software analytics tasks. However, current approaches for generating such tracking code face two key limitations: they either require substantial manual coding effort, or often fail to capture the full meaning behind user actions. To address this, in this paper, we take the first step toward applying large language models (LLMs) for the automated generation of tracking code. We present SmarTracker, a tool that first automatically instruments Android apps to track fine-grained UI-level user interactions. To capture the semantics of these interactions, SmarTracker analyzes Android app source code to extract rich contextual information. It then leverages an LLM to interpret user interaction contexts and generate tracking code that accurately captures the semantics of user behavior. We extensively evaluate SmarTracker on real-world Android apps. The result shows that it can reliably reconstruct user interactions with high accuracy, while preserving essential semantic details that reflect the underlying business logic of user behaviors. Runtime measurements further confirm that the system introduces minimal overhead. Overall, SmarTracker offers a practical and effective solution to integrating user behavior tracking into the mobile development workflow.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2603-2619"},"PeriodicalIF":6.0,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148288678","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach","authors":"Weichao Xu;Huaxin Pei;Jingxuan Yang;Yuchen Shi;Yi Zhang;Qianchuan Zhao","doi":"10.1109/TSE.2026.3703032","DOIUrl":"https://doi.org/10.1109/TSE.2026.3703032","url":null,"abstract":"Recent advances in decision-making policies have led to notable progress in various fields such as autonomous driving and robotics. However, testing these policies remains crucial with the existence of critical scenarios that may threaten their reliability. Despite ongoing research, challenges such as low testing efficiency and limited scenario diversity persist, primarily due to the complexity of the policies and their operating environments. To address these challenges, this paper proposes an adaptable Large Language Model (LLM)-driven adaptive testing framework to explore critical and diverse testing scenarios for decision-making policies. Specifically, we introduce a “generate-test-feedback” pipeline, leveraging templated prompt engineering to harness the world knowledge and reasoning abilities of LLMs. Additionally, a multi-scale scenario generation strategy is proposed to overcome the limitations of LLMs in making fine-grained adjustments, thereby further enhancing testing efficiency. Furthermore, during the feedback phase, a well-designed scenario evaluation module assesses the criticality and diversity of the generated scenarios for adaptive testing. The proposed LLM-driven method is extensively evaluated on five widely recognized benchmarks, demonstrating that our proposed method significantly outperforms baseline methods in uncovering both critical and diverse scenarios. These findings highlight the great promise of LLM-driven schemes in advancing the testing of decision-making policies.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"52 9","pages":"2721-2739"},"PeriodicalIF":6.0,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148985801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}