Journal of Systems and Software最新文献

{"title":"RAGVA: Engineering retrieval augmented generation-based virtual assistants in practice","authors":"Rui Yang ,&nbsp;Michael Fu ,&nbsp;Chakkrit Tantithamthavorn ,&nbsp;Chetan Arora ,&nbsp;Lisa Vandenhurk ,&nbsp;Joey Chua","doi":"10.1016/j.jss.2025.112436","DOIUrl":"10.1016/j.jss.2025.112436","url":null,"abstract":"<div><div>Retrieval-augmented generation (RAG)-based applications are gaining prominence due to their ability to leverage large language models (LLMs). These systems excel at combining retrieval mechanisms with generative capabilities, resulting in contextually relevant responses that enhance user experience. In particular, Transurban, a road operation company, replaced its rule-based virtual assistant (VA) with a RAG-based VA (RAGVA) to offer flexible customer interactions and support a wider range of scenarios. This paper presents an experience report from Transurban’s engineering team on building and deploying a RAGVA, offering a step-by-step guide for creating a conversational application and engineering a RAGVA. The report serves as a reference for future researchers and practitioners. While the engineering processes for traditional software applications are well-established, the development and evaluation of RAG-based applications are still in their early stages, with numerous emerging challenges remaining uncharted. To address this gap, we conduct a focus group study with Transurban practitioners regarding developing and evaluating their RAGVA. We identified eight challenges encountered by the engineering team and proposed eight future directions that should be explored to advance the development of RAG-based applications. This study contributes to the foundational understanding of a RAG-based conversational application and the emerging AI software engineering challenges it presents.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112436"},"PeriodicalIF":3.7,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143705032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring ethical values in software systems: A systematic literature review","authors":"Razieh Alidoosti ,&nbsp;Patricia Lago ,&nbsp;Maryam Razavian ,&nbsp;Antony Tang","doi":"10.1016/j.jss.2025.112430","DOIUrl":"10.1016/j.jss.2025.112430","url":null,"abstract":"<div><h3>Context:</h3><div>Ethics has attracted considerable attention in the field of software engineering. The prevalence of software-intensive systems and how they become an integral part of people’s lives makes it essential for software engineers to care about the potential impacts and injustice of software systems on people. The social and ethical implications on individuals and society play an important role in the study of software engineering ethics.</div></div><div><h3>Objective:</h3><div>The general aim of this paper is to study the state of the art in ethics in software engineering and to understand ethical considerations in software design and development.</div></div><div><h3>Method:</h3><div>We conducted a systematic literature review (SLR) with an initial sample of 623 articles, of which 85 articles were selected as primary studies.</div></div><div><h3>Result:</h3><div>We created a body of knowledge on stakeholders and software engineering ethical values. More specifically, we provided a stakeholders map to identify different types of system stakeholders and presented a value model to categorize ethical values and recognize value relations.</div></div><div><h3>Conclusion:</h3><div>This SLR sheds light on the state of software engineering ethics. It highlights challenges such as the identification of indirect stakeholders, the lack of attention to their concerns, and the need for a stakeholder classification. The review emphasizes the importance of considering stakeholders’ concerns to uncover ethical values and discusses methods for value extraction, with a focus on the value model. There are needs to address particular ethical values, resolve value conflicts, and operationalize ethical values in software design. These findings offer valuable insights for future research and practice, encouraging a comprehensive integration of ethical aspects into software engineering.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112430"},"PeriodicalIF":3.7,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143767765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CubeAgent: Efficient query-based video adversarial examples generation through deep reinforcement learning","authors":"Heyuan Shi ,&nbsp;Binqi Zeng ,&nbsp;Yu Zhan ,&nbsp;Rongkai Liu ,&nbsp;Yulin Yang ,&nbsp;Li Chen ,&nbsp;Chao Hu ,&nbsp;Ying Fu","doi":"10.1016/j.jss.2025.112437","DOIUrl":"10.1016/j.jss.2025.112437","url":null,"abstract":"<div><div>In commercial deep-learning-based video systems, testers utilize query-based methods to generate adversarial examples (AEs) and effectively uncover system vulnerabilities. The current research has primarily focused on selecting the key perturbation units, such as video patches, keyframes, and combinations of keyframes and regions, to add adversarial perturbation and generate AEs. Furthermore, deep reinforcement learning (DRL) frameworks have been utilized to model the results of sequence-based feedback to reduce query numbers. However, considering the pixels of spatial and temporal dimensions separately in the search process results in a large number of queries and intolerable failure rates for video AE generation. This paper proposes a new AEs perturbation unit called the “video cube”, which simultaneously extracts video pixels in neighbor frames and regions. We develop a new DRL framework called “CubeAgent”, which incorporates controllable policy actions for selecting the <em>number</em> and <em>index</em> of key video cubes segmented by time intervals. We conducted exhaustive experiments across diverse video DNN systems, utilizing the UCF101 and JESTER datasets, which conclusively demonstrated that CubeAgent can expedite the generation process by a factor of two, diminishing the average query count from 5,768 to 4,602, representing a 20% reduction, while simultaneously mitigating the average generation failure rate from 9% to 7%. The results show that CubeAgent improves the performance of adversarial example generation while achieving comparable.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112437"},"PeriodicalIF":3.7,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143705033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automated formal-specification-to-code trace links recovery using multi-dimensional similarity measures","authors":"Jiandong Li ,&nbsp;Shaoying Liu ,&nbsp;Zhi Jin","doi":"10.1016/j.jss.2025.112439","DOIUrl":"10.1016/j.jss.2025.112439","url":null,"abstract":"<div><div>Formal specification techniques are widely used in safety-critical system development, where precise alignment between the specification components and their implementation counterparts is essential for conformance verification and program maintenance. Existing methods for establishing these trace links are often inefficient, requiring manual effort, and automated approaches based on textual similarity suffer from low precision. In this paper, we propose a novel automated method that incorporates multi-dimensional attributes of formal specification components to improve trace link recovery. The underlying principle supporting our method is that the names, structures, and relationships of specification components are typically preserved in their implementation. Our method contains four steps: (1) identifying the components in both the specification and the code, (2) extracting the multi-dimensional attributes for both specification components and code components, (3) calculating their similarities, and (4) predicting trace links through ranking and comparing to a threshold. We evaluate our method across three projects and demonstrate that it performs better in average precision, recall and F1-score than existing text-based similarity techniques, including Latent Semantic Indexing, Vector Space Model, Word2Vec embeddings, and LLM-based embeddings. These results confirm that our approach provides a more effective and reliable solution for automatically establishing trace links between formal specifications and code.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112439"},"PeriodicalIF":3.7,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143682001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Architectural tactics and trade-offs for confidentiality","authors":"Cristian Orellana ,&nbsp;Hernán Astudillo","doi":"10.1016/j.jss.2025.112433","DOIUrl":"10.1016/j.jss.2025.112433","url":null,"abstract":"<div><div>Data protection has become a top priority, with blooming legal frameworks like the EU’s General Data Protection Regulation (GDPR) and others. Software architects need readily available information on existing alternative design decisions and their trade-offs regarding other quality attributes. Architectural tactic catalogs have been proposed to package architectural knowledge on design for specific quality attributes, but non for confidentiality per se and not as part of the traditional security C-I-A triad (Confidentiality, Integrity, Availability). This article surveys published confidentiality tactics; presents a stimulus-based scenario template; builds a new taxonomy that organizes existing and newly proposed confidentiality tactics; and aggregates expert knowledge about their trade-offs with other quality attributes. The taxonomy applicability is illustrated with the (actual) design rationale of an already published Internet of Things (IoT) system. Finally, the taxonomy usefulness for architectural decision-making is shown with an experimental study by 12 practitioners with varying degrees of experience. Although using the tactics catalog significantly improved <em>recall</em> and <em>F1-Score</em> for all subjects, this was especially true for seniors, suggesting that it enabled them to explore a broader solution space than just using previous knowledge. This new enriched taxonomy (1) extends the reach of architectural tactics to deal with confidentiality requirements and (2) provides guidance on their trade-offs regarding other quality attributes. This systematization increases the usefulness of architectural tactics as design techniques and facilitates their wider adoption by practicing architects for architectural decision-making.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112433"},"PeriodicalIF":3.7,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143738376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MidLog: An automated log anomaly detection method based on multi-head GRU","authors":"Wanli Yuan ,&nbsp;Shi Ying ,&nbsp;Xiaoyu Duan ,&nbsp;Hailong Cheng ,&nbsp;Yishi Zhao ,&nbsp;Jianga Shang","doi":"10.1016/j.jss.2025.112431","DOIUrl":"10.1016/j.jss.2025.112431","url":null,"abstract":"<div><div>Software systems typically utilize logs to record events that contain critical information. These logs are an indispensable data source for analyzing system anomalies. Large-scale log datasets have placed a tremendous burden on manually analyzing system logs as it is extremely time-consuming and error-prone. There have been many studies on log anomaly detection, whereas most existing deep learning methods lack flexibility and need auxiliary features to improve detection accuracy. We propose an automated anomaly detection method based on <u>m</u>ult<u>i</u>-hea<u>d</u> GRU for system <u>log</u>s, called MidLog. The core idea comes from the multi-head mechanism in Transformer. Multiple GRUs are used to learn normal sequence patterns hidden in system logs. Each GRU network is only responsible for learning a local sequence pattern. We conduct a global analysis of these local patterns to achieve log anomaly detection, which facilitates more accurate identification of log anomalies. The number of base models (GRUs) can be easily increased or decreased under the multi-head mechanism. Such a characteristic gives MidLog more flexibility and allows for a trade-off between detection accuracy and efficiency. Experiment results on public log datasets show that our method can achieve better detection accuracy compared with baseline methods.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112431"},"PeriodicalIF":3.7,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143682000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"BEACon-TD: Classifying Technical Debt and its types across diverse software projects issues using transformers","authors":"Karthik Shivashankar ,&nbsp;Mili Orucevic ,&nbsp;Maren Maritsdatter Kruke ,&nbsp;Antonio Martini","doi":"10.1016/j.jss.2025.112435","DOIUrl":"10.1016/j.jss.2025.112435","url":null,"abstract":"<div><div>Technical Debt (TD) identification in software projects issues is crucial for maintaining code quality, reducing long-term maintenance costs, and improving overall project health. This study advances TD identification in issues tracker using transformer-based models, addressing the critical need for accurate and efficient TD identification in large-scale software development.</div><div>Our methodology employs multiple binary classifiers for TD and its type, combined through ensemble learning, to enhance accuracy and robustness in detecting various forms of TD. We train and evaluate these models on a comprehensive dataset from GitHub Archive Issues (2015–2024), supplemented with industrial data validation.</div><div>We demonstrate that in-project fine-tuned transformer models significantly outperform task-specific fine-tuned models in TD classification, highlighting the importance of project-specific context in accurate TD identification. Our research also reveals the superiority of specialized binary classifiers over multi-class models for TD and its type identification, enabling more targeted debt resolution strategies. A comparative analysis shows that the smaller DistilRoBERTa model is more effective than larger language models like GPTs for TD classification tasks, especially after fine-tuning, offering insights into efficient model selection for specific TD detection tasks.</div><div>The study also assesses generalization capabilities using metrics such as MCC, AUC ROC, Recall, and F1 score, focusing on model effectiveness, fine-tuning impact, and relative performance. By validating our approach on out-of-distribution and real-world industrial datasets, we ensure practical applicability, addressing the diverse nature of software projects.</div><div>This research significantly enhances TD detection and offers a more nuanced understanding of TD types, contributing to improved software maintenance strategies in both academic and industrial settings. The release of our curated dataset aims to stimulate further advancements in TD classification research, ultimately enhancing software project outcomes and development practices by enabling early TD identification and management.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112435"},"PeriodicalIF":3.7,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143696636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Success with Agile Project Management: Looking back and into the future","authors":"Catarina Koudriachov,&nbsp;Carlos Tam,&nbsp;Manuela Aparicio","doi":"10.1016/j.jss.2025.112428","DOIUrl":"10.1016/j.jss.2025.112428","url":null,"abstract":"<div><div>We show what the influential factors and practical strategies are that contribute to agile project management success. The research model comprises three people-related factors (personal characteristics, team capability, and customer involvement), three technological factors (gamification, artificial intelligence, and marketing intelligence), and one dependent variable (agile project management success). Based on 143 questionnaire responses, our findings reaffirm the positive impact of personal characteristics and customer involvement while challenging the roles of gamification and team capability, suggesting that their effects are more context-dependent than previously thought. Our findings also highlight that agile project management success depends on the interplay between remote work and team capability, with strong team skills being highly important for agile methodologies, especially in traditional office settings.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112428"},"PeriodicalIF":3.7,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143681999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Poisoned source code detection in code models","authors":"Ehab Ghannoum,&nbsp;Mohammad Ghafari","doi":"10.1016/j.jss.2025.112384","DOIUrl":"10.1016/j.jss.2025.112384","url":null,"abstract":"<div><div>Deep learning models have gained popularity for conducting various tasks involving source code. However, their black-box nature raises concerns about potential risks. One such risk is a poisoning attack, where an attacker intentionally contaminates the training set with malicious samples to mislead the model’s predictions in specific scenarios. To protect source code models from poisoning attacks, we introduce CodeGarrison (CG), a hybrid deep-learning model that relies on code embeddings to identify poisoned code samples. We evaluated CG against the state-of-the-art technique ONION for detecting poisoned samples generated by DAMP, MHM, ALERT, as well as a novel poisoning technique named CodeFooler. Results showed that CG significantly outperformed ONION with an accuracy of 93.5%. We also tested CG’s robustness against unknown attacks, achieving an average accuracy of 85.6% in identifying poisoned samples across the four attacks mentioned above.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112384"},"PeriodicalIF":3.7,"publicationDate":"2025-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143620275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"IoT systems testing: Taxonomy, empirical findings, and recommendations","authors":"Jean Baptiste Minani ,&nbsp;Yahia El Fellah ,&nbsp;Fatima Sabir ,&nbsp;Naouel Moha ,&nbsp;Yann-Gaël Guéhéneuc ,&nbsp;Martin Kuradusenge ,&nbsp;Tomoaki Masuda","doi":"10.1016/j.jss.2025.112408","DOIUrl":"10.1016/j.jss.2025.112408","url":null,"abstract":"<div><div>The Internet of Things (IoT) is reshaping our lives, increasing the need for thorough pre-deployment testing. However, traditional software testing may not address the testing requirements of IoT systems, leading to quality challenges. A specific testing taxonomy is crucial, yet no widely recognized taxonomy exists for IoT system testing. We introduced an IoT-specific testing taxonomy that categorizes aspects of IoT systems testing into seven distinct categories. We mined testing aspects from 83 primary studies in IoT systems testing and built an initial taxonomy. This taxonomy was refined and validated through two rounds of surveys involving 16 and then 204 IoT industry practitioners. We assessed its effectiveness by conducting an empirical evaluation on two separate IoT systems, each involving 12 testers. Our findings categorize seven testing aspects: (1) testing objectives, (2) testing tools and artifacts, (3) testers, (4) testing stage, (5) testing environment, (6) Object Under Test (OUT) and metrics, and (7) testing approaches. The evaluation showed that testers equipped with the taxonomy could more effectively identify diverse test cases and scenarios. Additionally, we recommend new research opportunities to enhance the testing of IoT systems.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"226 ","pages":"Article 112408"},"PeriodicalIF":3.7,"publicationDate":"2025-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143609961","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}