Journal of Systems and Software最新文献

{"title":"Into the ML-Universe: An improved classification and characterization of machine-learning projects","authors":"Vincenzo De Martino,&nbsp;Gilberto Recupito,&nbsp;Giammaria Giordano,&nbsp;Filomena Ferrucci,&nbsp;Dario Di Nucci,&nbsp;Fabio Palomba","doi":"10.1016/j.jss.2025.112471","DOIUrl":"10.1016/j.jss.2025.112471","url":null,"abstract":"<div><div>The prominence of Machine Learning (ML) systems led to the rise of Software Engineering for Artificial Intelligence (SE4AI), which addresses the unique engineering challenges of these systems. Researchers in SE4AI engage with three primary types of ML projects: those that apply ML techniques, those that develop new ML methodologies, and those that provide support tools and libraries. Current classification schemas distinguish ML projects based on their purpose and engineering quality, yet they miss a fine-grained classification of their nature and purpose. In this paper, we propose a novel, tool-supported automated classification schema for ML projects, coined <span><strong><u>M</u></strong>achine learning <strong><u>A</u></strong>utomated <strong><u>R</u></strong>ule-based Classification <strong><u>K</u></strong>it</span> (MARK), that builds on top of the work by Gonzalez et al. to refine the classification of applied ML projects into <em>‘ML-Model Consumers,’ ‘ML-Model Producers,’</em> and <em>‘ML-Model Producers &amp; Consumers.’</em> We evaluated MARK through two empirical studies. The first assessed its classification accuracy across 4,603 ML projects from two datasets. The second analyzed repository metrics, such as community engagement, activity, and structure, to demonstrate MARK’s potential in identifying trends and characteristics unique to each project type. Our findings indicate high F1-scores for our classifier, particularly for <em>‘ML-Model Producer’</em> projects, though challenges remain for <em>‘ML-Model Consumer’</em> classification. Significant differences in repository metrics among the classified projects highlight the usefulness of MARK, offering insights for researchers studying the socio-technical dynamics of ML projects.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112471"},"PeriodicalIF":3.7,"publicationDate":"2025-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144579871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"It all starts with structure: investigating learning dynamics in large-scale agile software development","authors":"Muhammad Ovais Ahmad ,&nbsp;Hadi Ghanbari ,&nbsp;Tomas Gustavsson ,&nbsp;Bikesh Raj Upreti","doi":"10.1016/j.jss.2025.112561","DOIUrl":"10.1016/j.jss.2025.112561","url":null,"abstract":"<div><div>Agile software development (ASD) methods have increasingly been used in large-scale software development projects. While ASD emphasizes the importance of social interactions between practitioners for continuous reflection and knowledge sharing, these learning activities become incredibly challenging in large-scale projects. Drawing on well-established theoretical concepts, we posit that learning in large-scale ASD projects requires a suitable environment that empowers practitioners to openly and frequently engage in social interactions, which are essential for reflection and knowledge sharing. We hypothesize that several team-level factors shape individuals’ perceptions about the learning environment and learning activities in their projects, ultimately influencing their learning behavior. To test our model, we collected survey responses from practitioners working in large-scale ASD projects in five Swedish companies (<em>N</em> = 159). The data was analyzed using confirmatory factor analysis (CFA) and structural equation modeling (SEM). The results show that team structure plays a crucial role in promoting team cohesion and reflexivity, which, alongside knowledge sharing, contribute to the learning process. Our study provides ASD development research with a theoretically informed understanding of the interrelationship between the learning environment and learning activities in large-scale ASD. Our results guide practitioners in fostering suitable learning environments and enhancing learning in large-scale ASD.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112561"},"PeriodicalIF":3.7,"publicationDate":"2025-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learning never stops: Improving software vulnerability type identification via incremental learning","authors":"Jiacheng Xue ,&nbsp;Xiang Chen ,&nbsp;Zhanqi Cui ,&nbsp;Yong Liu","doi":"10.1016/j.jss.2025.112544","DOIUrl":"10.1016/j.jss.2025.112544","url":null,"abstract":"<div><div>As new vulnerabilities are continuously discovered, software vulnerability type identification (SVTI) data is dynamic. Moreover, SVTI data often exhibits a long-tailed distribution, where some vulnerability types (i.e., head classes) have numerous samples, while rare ones (i.e., tail classes) have very few. These issues present challenges for SVTI, such as catastrophic forgetting when learning new data and poor performance for rare vulnerability types. To address these challenges, we propose an approach <em>VulTypeIL</em>. Specifically, for incremental learning, we employ a hybrid replay strategy and a regularization strategy with EWC to alleviate the catastrophic forgetting issue. We also integrate focal loss and label smooth cross-entropy loss to tackle the long-tailed distribution issue. For model construction, we customize the verbalizer and hybrid prompt by fusing the Vulnerability code and description. Then, we perform prompt tuning on the pre-trained model CodeT5. To evaluate the effectiveness of <em>VulTypeIL</em>, we construct a large-scale SVTI dataset containing 6,269 vulnerabilities from 992 real-world projects. Our experimental results demonstrate that <em>VulTypeIL</em> outperforms state-of-the-art baselines (such as VulExplainer and LIVABLE) with a significant improvement. The ablation studies further confirm the effectiveness of key component settings (such as the incremental learning setting and long-tailed learning setting) in our approach.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112544"},"PeriodicalIF":3.7,"publicationDate":"2025-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144562944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Managing technical debt in a multidisciplinary data intensive software team: An observational case study","authors":"Ulrike M. Graetsch ,&nbsp;Rashina Hoda ,&nbsp;Hourieh Khalajzadeh ,&nbsp;Mojtaba Shahin ,&nbsp;John Grundy","doi":"10.1016/j.jss.2025.112546","DOIUrl":"10.1016/j.jss.2025.112546","url":null,"abstract":"<div><h3>Context:</h3><div>There is an increase in the investment and development of data-intensive (DI) solutions — systems that manage large amounts of data. Without careful management, this growing investment will also grow associated technical debt (TD). Delivery of DI solutions requires a multidisciplinary skill set, but there is limited knowledge about how multidisciplinary teams develop DI systems and manage TD.</div></div><div><h3>Objective:</h3><div>This research contributes empirical, practice based insights about multidisciplinary DI team TD management practices.</div></div><div><h3>Method:</h3><div>This research was conducted as an exploratory observation case study. We used socio-technical grounded theory (STGT) <em>for data analysis</em> to develop concepts and categories that articulate TD and TDs debt management practices.</div></div><div><h3>Results:</h3><div>We identify TD that the DI team deals with, in particular technical data components debt and pipeline debt. We explain how the team manages the TD, assesses TD, what TD treatments they consider and how they implement TD treatments to fit sprint capacity constraints.</div></div><div><h3>Conclusion:</h3><div>We align our findings to existing TD and TDM taxonomies, discuss their implications and highlight the need for new implementation patterns and tool support for multidisciplinary DI teams.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112546"},"PeriodicalIF":3.7,"publicationDate":"2025-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144569938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Analyzing the dependability of Large Language Models for code clone generation","authors":"Azeeza Eagal,&nbsp;Kathryn T. Stolee,&nbsp;John-Paul Ore","doi":"10.1016/j.jss.2025.112548","DOIUrl":"10.1016/j.jss.2025.112548","url":null,"abstract":"<div><div>The ability to generate multiple equivalent versions of the same code segment across different programming languages and within the same language is valuable for code translation, language migration, and code comprehension in education. However, current avenues for generating code clones – through manual creation or specialized software tools – often fail to consistently generate a variety of behaviorally equivalent code clones. Large Language Models (LLMs) offer a promising solution by leveraging their extensive training on diverse codebases to automatically generate code. Unlike traditional methods, LLMs can produce code across a wide variety of programming languages with minimal user effort. Using LLMs for code clone generation could significantly reduce the time and resources needed to create code clones while enhancing their syntactic diversity.</div><div>In this quantitative empirical study, we investigate the dependability of LLMs as potential generators of code clones. We gathered equivalent code solutions (i.e., behavioral clones) in C++, Java, and Python from thirty-six programming problems from the well-known technical interview practice platform, LeetCode. We query OpenAI’s GPT-3.5, GPT-4, and CodeLlama to generate code clones of the LeetCode solutions. We measure the behavioral equivalence of the LLM-generated clones using a behavioral similarity clustering technique inspired by the code clone detection tool, Simion-based Language Agnostic Code Clones (SLACC). This study reveals that, despite LLMs demonstrating the potential for code generation, their capacity to consistently generate syntactically diverse but behaviorally equivalent code clones is limited. At lower temperature settings, LLMs are more successful in producing behaviorally consistent, syntactically similar code clones within the same language. However, for cross-language cloning tasks and at higher temperature settings and programming difficulties, LLMs introduce greater syntactic diversity and lead to higher rates of compilation and runtime errors, resulting in a decline in behavioral consistency. These findings indicate a need for further quality assurance measures for the use of LLMs for code clone generation. All the data and scripts associated with this paper can be found <span><span>https://zenodo.org/records/14968618</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112548"},"PeriodicalIF":3.7,"publicationDate":"2025-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144623507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"LLMs and Stack Overflow discussions: Reliability, impact, and challenges","authors":"Leuson Da Silva ,&nbsp;Jordan Samhi ,&nbsp;Foutse Khomh","doi":"10.1016/j.jss.2025.112541","DOIUrl":"10.1016/j.jss.2025.112541","url":null,"abstract":"<div><div>Since its release in November 2022, ChatGPT has shaken up Stack Overflow, the premier platform for developers’ queries on programming and software development. Demonstrating an ability to generate instant, human-like responses to technical questions, ChatGPT has ignited debates within the developer community about the evolving role of human-driven platforms in the age of generative AI. Two months after ChatGPT’s release, Meta released its answer with its own Large Language Model (LLM) called LLaMA: <em>the race was on</em>. We conducted an empirical study analyzing questions from Stack Overflow and using these LLMs to address them. This way, we aim to quantify the reliability of LLMs’ answers and their potential to replace Stack Overflow in the long term; identify and understand why LLMs fail; measure users’ activity evolution with Stack Overflow over time; and compare LLMs together. Our empirical results are unequivocal: <em>ChatGPT and LLaMA challenge human expertise, yet do not outperform it for some domains</em>, while a significant decline in user posting activity has been observed. Furthermore, we also discuss the impact of our findings regarding the usage and development of new LLMs and provide guidelines for future challenges faced by users and researchers.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112541"},"PeriodicalIF":3.7,"publicationDate":"2025-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144569939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A practitioner survey on Requirements Technical Debt Quantification","authors":"Judith Perera ,&nbsp;Ewan Tempero ,&nbsp;Yu-Cheng Tu ,&nbsp;Kelly Blincoe","doi":"10.1016/j.jss.2025.112538","DOIUrl":"10.1016/j.jss.2025.112538","url":null,"abstract":"<div><div>Requirements Technical Debt (RTD) is a phenomenon borrowed from the Technical Debt literature that captures the consequences of sub-optimal decisions made concerning software requirements. We report on a survey conducted to understand industry practices and perceptions about RTD quantification.</div><div>The survey instrument was designed based on prior work, the RTD Quantification Model (RTDQM), which captures RTD quantification conceptually and serves as a reference point. Our survey employs the Critical Incident Technique (CIT), inquiring practitioners about what RTD instances they encountered. Follow-up questions focused on understanding whether practitioners fixed such RTD instances and whether they quantified model concepts such as the Cost of fixing, the Benefit of fixing, and the Consequences of not fixing. We also sought their opinions on whether quantification supports decision-making.</div><div>Our findings suggest that the Benefit of fixing RTD is the concept agreed by most practitioners that it supports decision-making, and is quantified in practice. Practitioners’ preferences regarding the concepts to quantify seem to differ based on the different RTD instances. Survey findings also suggest that the company and individual perspectives regarding the quantification of the concepts differ. Our findings reveal future research avenues that warrant deeper conversations with the industry.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112538"},"PeriodicalIF":3.7,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144587674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the adoption of software bill of materials in open-source software projects","authors":"Sabato Nocera ,&nbsp;Simone Romano ,&nbsp;Massimiliano Di Penta ,&nbsp;Rita Francese ,&nbsp;Giuseppe Scanniello","doi":"10.1016/j.jss.2025.112540","DOIUrl":"10.1016/j.jss.2025.112540","url":null,"abstract":"<div><div>A <em>Software Bill of Materials</em> (<em>SBOM</em>) formally lists the open-source and proprietary components that constitute a software product, including their licenses, versions, vendors, vulnerabilities, and supply chain relationships. SBOMs enable software producers and consumers to gain visibility into the software supply chain and monitor the risks associated with software security, licensing, and more. This paper presents the results of an exploratory mining study investigating the adoption of SBOMs by open-source software projects. To that end, we mined <em>GitHub</em> and identified 186 public software repositories using SBOM generation tools owned by <em>SPDX</em> and <em>CycloneDX</em>. Although the adoption of SBOMs is low, it is increasing. Moreover, SBOMs are under version control or available in public release versions of less than half the software projects analyzed. Finally, only a limited fraction of SBOMs contain minimum/recommended information, and some SBOMs are also uncompliant with existing SBOM standards. Our study reveals that software producers are paying more attention to SBOMs, but even so, these may be incomplete. We urge software producers to adopt SBOMs and meet the new software supply chain standards. As for researchers, we foster further investigations on adopting SBOMs and their correct use.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112540"},"PeriodicalIF":3.7,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144623670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"InvAASTCluster: On Applying Invariant-Based Program Clustering to Introductory Programming Assignments","authors":"Pedro Orvalho ,&nbsp;Mikoláš Janota ,&nbsp;Vasco Manquinho","doi":"10.1016/j.jss.2025.112481","DOIUrl":"10.1016/j.jss.2025.112481","url":null,"abstract":"<div><div>Due to the vast number of students enrolled in programming courses, there has been an increasing number of automated program repair techniques focused on introductory programming assignments (<span>IPAs</span>). Typically, such techniques use program clustering to take advantage of previous correct student implementations to repair a new incorrect submission. These repair techniques use clustering methods since analyzing all available correct submissions to repair a program is not feasible. However, conventional clustering methods rely on program representations based on features such as abstract syntax trees (<span>ASTs</span>), syntax, control flow, and data flow.</div><div>This paper proposes <span>InvAASTCluster</span>, a novel approach for program clustering that uses dynamically generated program invariants to cluster semantically equivalent <span>IPAs</span>. <span>InvAASTCluster</span>’s program representation uses a combination of the program’s semantics, through its invariants, and its structure through its anonymized abstract syntax tree (<span>AASTs</span>). Invariants denote conditions that must remain true during program execution, while <span>AASTs</span> are <span>ASTs</span> devoid of variable and function names, retaining only their types. Our experiments show that the proposed program representation outperforms syntax-based representations when clustering a set of correct <span>IPAs</span>. Furthermore, we integrate <span>InvAASTCluster</span> into a state-of-the-art clustering-based program repair tool. Our results show that <span>InvAASTCluster</span> advances the current state-of-the-art when used by clustering-based repair tools by repairing around 13% more students’ programs, in a shorter amount of time.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112481"},"PeriodicalIF":3.7,"publicationDate":"2025-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144579858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Domain-Driven Design in software development: A systematic literature review on implementation, challenges, and effectiveness","authors":"Ozan Özkan ,&nbsp;Önder Babur ,&nbsp;Mark van den Brand","doi":"10.1016/j.jss.2025.112537","DOIUrl":"10.1016/j.jss.2025.112537","url":null,"abstract":"<div><h3>Context:</h3><div>Domain-Driven Design (DDD) has gained significant attention in software development for its potential to address complex software challenges, particularly in the areas of system refactoring, reimplementation, and adoption. Using domain knowledge, DDD aims to solve complex business problems effectively.</div></div><div><h3>Objective:</h3><div>This SLR aims to provide an analysis of existing research on DDD in software development, paint a picture of DDD in solving software problems, identify the challenges encountered during its application and explore the results of these studies.</div></div><div><h3>Method:</h3><div>We systematically selected 36 peer reviewed studies and conducted quantitative and qualitative analyzes to synthesize the findings.</div></div><div><h3>Results:</h3><div>DDD has effectively improved software systems, with its key concepts. The application of DDD in microservices has gained prominence for its ability to facilitate system decomposition. Some studies lacked empirical evaluations, highlighting challenges in onboarding and the need for expertise.</div></div><div><h3>Conclusion:</h3><div>Adopting DDD benefits software development, involving stakeholders such as engineers, architects, managers, and domain experts. More empirical evaluations and open discussions on challenges are needed. Collaboration between academia and industry advances the adoption and transfer of knowledge of DDD in projects.</div></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"230 ","pages":"Article 112537"},"PeriodicalIF":3.7,"publicationDate":"2025-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144563745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}