IEEE Transactions on Software Engineering最新文献

{"title":"2024 Reviewers List","authors":"","doi":"10.1109/TSE.2024.3525202","DOIUrl":"https://doi.org/10.1109/TSE.2024.3525202","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 1","pages":"320-324"},"PeriodicalIF":6.5,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10837582","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards Improving the Performance of Comment Generation Models by Using Bytecode Information","authors":"Yuan Huang;Jinbo Huang;Xiangping Chen;Zibin Zheng","doi":"10.1109/TSE.2024.3523713","DOIUrl":"10.1109/TSE.2024.3523713","url":null,"abstract":"Code comment plays an important role in program understanding, and a large number of automatic comment generation methods have been proposed in recent years. To get a better effect of generating comments, many studies try to extract a variety of information (e.g., code tokens, AST traverse sequence, APIs call sequence) from source code as model input. In this study, we found that the bytecode compiled from the source code can provide useful information for comment generation, hence we propose to use the information from bytecode to assist the comment generation. Specifically, we extract the control flow graph (CFG) from the bytecode and propose a serialization method to obtain the CFG sequence that preserves the program structure. Then, we discuss three methods for introducing bytecode information for different models. We collected 390,000 Java methods from the maven repository, and created a dataset of 101,124 samples after deduplication and preprocessing to evaluate our method. The results show that introducing the information extracted from the bytecode can improve the BLEU-4 of 7 comment generation models.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 2","pages":"503-520"},"PeriodicalIF":6.5,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evidence-based Software Engineering Guidelines Revisited","authors":"Shari Lawrence Pfleeger, Barbara Kitchenham","doi":"10.1109/tse.2025.3526730","DOIUrl":"https://doi.org/10.1109/tse.2025.3526730","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"8 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142936793","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Accuracy Can Lie: On the Impact of Surrogate Model in Configuration Tuning","authors":"Pengzhou Chen;Jingzhi Gong;Tao Chen","doi":"10.1109/TSE.2025.3525955","DOIUrl":"10.1109/TSE.2025.3525955","url":null,"abstract":"To ease the expensive measurements during configuration tuning, it is natural to build a surrogate model as the replacement of the system, and thereby the configuration performance can be cheaply evaluated. Yet, a stereotype therein is that the higher the model accuracy, the better the tuning result would be, or vice versa. This “accuracy is all” belief drives our research community to build more and more accurate models and criticize a tuner for the inaccuracy of the model used. However, this practice raises some previously unaddressed questions, e.g., are the model and its accuracy really that important for the tuning result? Do those somewhat small accuracy improvements reported (e.g., a few % error reduction) in existing work really matter much to the tuners? What role does model accuracy play in the impact of tuning quality? To answer those related questions, in this paper, we conduct one of the largest-scale empirical studies to date—running over the period of 13 months <inline-formula><tex-math>$24times 7$</tex-math></inline-formula>—that covers 10 models, 17 tuners, and 29 systems from the existing works while under four different commonly used metrics, leading to 13,612 cases of investigation. Surprisingly, our key findings reveal that the accuracy can lie: there are a considerable number of cases where higher accuracy actually leads to no improvement in the tuning outcomes (up to 58% cases under certain setting), or even worse, it can degrade the tuning quality (up to 24% cases under certain setting). We also discover that the chosen models in most proposed tuners are sub-optimal and that the required % of accuracy change to significantly improve tuning quality varies according to the range of model accuracy. Deriving from the fitness landscape analysis, we provide in-depth discussions of the rationale behind, offering several lessons learned as well as insights for future opportunities. Most importantly, this work poses a clear message to the community: we should take one step back from the natural “accuracy is all” belief for model-based configuration tuning.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 2","pages":"548-580"},"PeriodicalIF":6.5,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10832565","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142936245","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Retrospective on the Source Code Control System","authors":"Marc J. Rochkind","doi":"10.1109/tse.2024.3524947","DOIUrl":"https://doi.org/10.1109/tse.2024.3524947","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"14 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2025-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142924713","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Holistic Approach to Design Understanding Through Concept Explanation","authors":"Hongzhou Fang;Yuanfang Cai;Ewan Tempero;Rick Kazman;Yu-Cheng Tu;Jason Lefever;Ernst Pisch","doi":"10.1109/TSE.2024.3522973","DOIUrl":"10.1109/TSE.2024.3522973","url":null,"abstract":"Complex software systems consist of multiple overlapping design structures, such as abstractions, features, crosscutting concerns, or patterns. This is similar to how a human body has multiple interacting subsystems, such as respiratory, digestive, or circulatory. Unlike in the medical domain, software designers do not have an effective way to distinguish, visualize, comprehend, and analyze these interleaving design structures. As a result, developers often struggle through the maze of source code. In this paper, we present an <italic>Automated Concept Explanation</i> (ACE) framework that automatically extracts and categorizes major concepts from source code based on the roles that files play in design structures and their topic frequencies. Based on these categorized concepts, ACE recovers four categories of high-level design models using different algorithms and generates a natural language explanation for each. To assess if and how ACE can help developers better understand design structures, we conducted an empirical study where two groups of graduate students were assigned three design comprehension tasks: identifying feature-related files, identifying dependencies among features, and identifying design patterns used, in an open-source project. The results reveal that the students who used ACE can accomplish these tasks much faster and more accurately, and they acknowledged the usefulness of the categorized concepts and structures, multi-type high-level model visualization, and natural language explanations.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 2","pages":"449-465"},"PeriodicalIF":6.5,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models","authors":"Yuheng Huang;Jiayang Song;Zhijie Wang;Shengming Zhao;Huaming Chen;Felix Juefei-Xu;Lei Ma","doi":"10.1109/TSE.2024.3519464","DOIUrl":"10.1109/TSE.2024.3519464","url":null,"abstract":"The recent performance leap of Large Language Models (LLMs) opens up new opportunities across numerous industrial applications and domains. However, the potential erroneous behavior (e.g., the generation of misinformation and hallucination) has also raised severe concerns for the trustworthiness of LLMs, especially in safety-, security- and reliability-sensitive industrial scenarios, potentially hindering real-world adoptions. While uncertainty estimation has shown its potential for interpreting the prediction risks made by classic machine learning (ML) models, the unique characteristics of recent LLMs (e.g., adopting self-attention mechanism as its core, very large-scale model size, often used in generative contexts) pose new challenges for the behavior analysis of LLMs. Up to the present, little progress has been made to better understand whether and to what extent uncertainty estimation can help characterize the capability boundary of an LLM, to counteract its undesired behavior, which is considered to be of great importance with the potential wide-range applications of LLMs across industry domains. To bridge the gap, in this paper, we initiate an early exploratory study of the risk assessment of LLMs from the lens of uncertainty. In particular, we conduct a large-scale study with as many as twelve uncertainty estimation methods and eight general LLMs on four NLP tasks and seven programming-capable LLMs on two code generation tasks to investigate to what extent uncertainty estimation techniques could help characterize the prediction risks of LLMs. Our findings confirm the potential of uncertainty estimation for revealing LLMs’ uncertain/non-factual predictions. The insights derived from our study can pave the way for more advanced analysis and research on LLMs, ultimately aiming at enhancing their trustworthiness.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 2","pages":"413-429"},"PeriodicalIF":6.5,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Refactoring Microservices to Microservices in Support of Evolutionary Design","authors":"Chenxing Zhong;Shanshan Li;He Zhang;Huang Huang;Lanxin Yang;Yuanfang Cai","doi":"10.1109/TSE.2024.3523487","DOIUrl":"10.1109/TSE.2024.3523487","url":null,"abstract":"<italic>Evolutionary design</i> is a widely accepted practice for defining microservice boundaries. It is performed through a sequence of incremental refactoring tasks (we call it <italic>“microservice refactoring”</i>), each restructuring only part of a microservice system (<italic>a.k.a., refactoring part</i>) into well-defined services for improving the architecture in a controlled manner. Despite its popularity in practice, microservice refactoring suffers from insufficient methodological support. While there are numerous studies addressing similar software design tasks, <italic>i</i>.<italic>e</i>., software remodularization and microservitization, their approaches prove inadequate when applied to microservice refactoring. Our analysis reveals that their approaches may even degrade the entire architecture in microservice refactoring, as they only optimize the refactoring part in such applications, but neglect the relationships between the refactoring part and the remaining system. As the first response to the need, <italic>Micro2Micro</i> is proposed to re-partition the refactoring part while optimizing three quality objectives including the interdependence between the refactoring and non-refactoring parts. In addition, it allows architects to intervene in the decision-making process by interactively incorporating their knowledge into the iterative search for optimal refactoring solutions. An empirical study on 13 open-source projects of different sizes shows that the solutions from <italic>Micro2Micro</i> perform well and exhibit quality improvement with an average up to 45% to the original architecture. Users of <italic>Micro2Micro</i> found the suggested solutions highly satisfactory. They acknowledge the advantages in terms of infusing human intelligence into decisions, providing immediate quality feedback, and quick exploration capability.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 2","pages":"484-502"},"PeriodicalIF":6.5,"publicationDate":"2024-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142908423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Retrospective on Developing Code Clone Detector CCFinder and Its Impact","authors":"Toshihiro Kamiya, Shinji Kusumoto, Katsuro Inoue","doi":"10.1109/tse.2024.3523370","DOIUrl":"https://doi.org/10.1109/tse.2024.3523370","url":null,"abstract":"","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"29 1","pages":""},"PeriodicalIF":7.4,"publicationDate":"2024-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142888334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Do Chase Your Tail! Missing Key Aspects Augmentation in Textual Vulnerability Descriptions of Long-Tail Software Through Feature Inference","authors":"Linyi Han;Shidong Pan;Zhenchang Xing;Jiamou Sun;Sofonias Yitagesu;Xiaowang Zhang;Zhiyong Feng","doi":"10.1109/TSE.2024.3523284","DOIUrl":"10.1109/TSE.2024.3523284","url":null,"abstract":"Augmenting missing key aspects in Textual Vulnerability Descriptions (TVDs) is crucial for effective vulnerability analysis. For instance, in TVDs, key aspects include <italic>Attack Vector</i>, <italic>Vulnerability Type</i>, among others. These key aspects help security engineers understand and address the vulnerability in a timely manner. For software with a large user base (non-long-tail software), augmenting these missing key aspects has significantly advanced vulnerability analysis and software security research. However, software instances with a limited user base (long-tail software) often get overlooked due to inconsistency software names, TVD limited avaliability, and domain-specific jargon, which complicates vulnerability analysis and software repairs. In this paper, we introduce a novel software feature inference framework designed to augment the missing key aspects of TVDs for long-tail software. Firstly, we tackle the issue of non-standard software names found in community-maintained vulnerability databases by cross-referencing government databases with Common Vulnerabilities and Exposures (CVEs). Next, we employ Large Language Models (LLMs) to generate the missing key aspects. However, the limited availability of historical TVDs restricts the variety of examples. To overcome this limitation, we utilize the Common Weakness Enumeration (CWE) to classify all TVDs and select cluster centers as representative examples. To ensure accuracy, we present Natural Language Inference (NLI) models specifically designed for long-tail software. These models identify and eliminate incorrect responses. Additionally, we use a wiki repository to provide explanations for proprietary terms. Our evaluations demonstrate that our approach significantly improves the accuracy of augmenting missing key aspects of TVDs for log-tail software from 0.27 to 0.56 (+107%). Interestingly, the accuracy of non-long-tail software also increases from 64% to 71%. As a result, our approach can be useful in various downstream tasks that require complete TVD information.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 2","pages":"466-483"},"PeriodicalIF":6.5,"publicationDate":"2024-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142888021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}