Journal of Systems and Software最新文献

{"title":"Feature-oriented test case selection and prioritization during the evolution of highly-configurable systems","authors":"Willian D.F. Mendonça ,&nbsp;Wesley K.G. Assunção ,&nbsp;Silvia R. Vergilio","doi":"10.1016/j.jss.2024.112157","DOIUrl":"10.1016/j.jss.2024.112157","url":null,"abstract":"<div><p>Testing <em>Highly Configurable Systems (HCSs)</em> is a challenging task, especially in an evolution scenario where features are added, changed, or removed, which hampers test case selection and prioritization. Existing work is usually based on the variability model, which is not always available or updated. Yet, the few existing approaches rely on links between test cases and changed files (or lines of code), not considering how features are implemented, usually spread over several and unchanged files. To overcome these limitations, we introduce <span>FeaTestSelPrio</span>, a feature-oriented test case selection and prioritization approach for HCSs. The approach links test cases to feature implementations, using HCS pre-processor directives, to select test cases based on features affected by changes in each commit. After, the test cases are prioritized according to the number of features they cover. Our approach selects a greater number of tests and takes longer to execute than a changed-file-oriented approach, used as baseline, but <span>FeaTestSelPrio</span> performs better regarding detected failures. By adding the approach execution time to the execution time of the selected test cases, we reached a reduction of <span><math><mo>≈</mo></math></span>50%, in comparison with retest-all. The prioritization step allows reducing the average test budget in 86% of the failed commits.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"217 ","pages":"Article 112157"},"PeriodicalIF":3.7,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141951082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluation of visual property specification languages based on practical model-checking experience","authors":"Antti Pakonen ,&nbsp;Igor Buzhinsky ,&nbsp;Valeriy Vyatkin","doi":"10.1016/j.jss.2024.112153","DOIUrl":"10.1016/j.jss.2024.112153","url":null,"abstract":"<div><p>Formal verification methods like model checking can provide mathematical proofs of design correctness, so their use is justified in applications where safety or reliability requirements are high. A key challenge for the wider adoption of model checking is the effort and expertise needed in formalizing functional requirements into verifiable properties. A particular challenge in specifying formal properties for industrial instrumentation and control (I&amp;C) logics is accounting for the sequencing and timing issues that arise from, e.g., the dynamic behavior of the plant being controlled. In this paper, we evaluate different visual property specification languages that are aimed at making formal methods more accessible. We have collected 3923 formal properties from practical model checking projects in the nuclear and rail traffic industries and identified the most commonly occurring types of properties. Based on the sample data, a real-world example logic, and our practical experience, we identify requirements for a user-friendly property specification language most suited for our specific domain of industrial I&amp;C.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112153"},"PeriodicalIF":3.7,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224001985/pdfft?md5=46204c3debb73585b8f0184dffbea2dd&pid=1-s2.0-S0164121224001985-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141638305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A systematic literature review on characteristics of the front-end phase of agile software development projects and their connections to project success","authors":"Magne Jørgensen","doi":"10.1016/j.jss.2024.112155","DOIUrl":"10.1016/j.jss.2024.112155","url":null,"abstract":"<div><h3>Context</h3><p>Software development of new products and services often involves a front-end phase where user needs are analysed, costs and benefits are estimated, and initial plans are created.</p></div><div><h3>Goal</h3><p>This study aims to learn more about how the introduction of agile software development has affected practices and outcomes related to cost and benefit estimation in this front-end phase and to understand better what would improve this phase.</p></div><div><h3>Method</h3><p>We identified, reviewed and aggregated the results from 42 relevant research articles by searching literature databases and snowballing relevant articles.</p></div><div><h3>Results</h3><p>The front-end phase of agile was found to be, on average, similar and just as comprehensive as that of non-agile software development. This may be unfortunate, given the finding that more successful agile software development is connected with less detail in cost estimation and planning-related activities. A less comprehensive front-end phase may be especially beneficial for low-risk agile software development.</p></div><div><h3>Conclusion</h3><p>The results of this review suggest that agile principles, so far, have had a limited influence on the front-end phase. We recommend more flexibility and context-dependency in how the front-end phase of agile software development is conducted, including less comprehensive estimation and planning activities for low-risk software development contexts.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112155"},"PeriodicalIF":3.7,"publicationDate":"2024-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141690673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"AML: An accuracy metric model for effective evaluation of log parsing techniques","authors":"Issam Sedki,&nbsp;Abdelwahab Hamou-Lhadj,&nbsp;Otmane Ait Mohamed","doi":"10.1016/j.jss.2024.112154","DOIUrl":"10.1016/j.jss.2024.112154","url":null,"abstract":"<div><p>Logs are essential for the maintenance of large software systems. Software engineers often analyze logs for debugging, root cause analysis, and anomaly detection tasks. Logs, however, are partly structured, making the extraction of useful information from massive log files a challenging task. Recently, many log parsing techniques have been proposed to automatically extract log templates from unstructured log files. These parsers, however, are evaluated using different accuracy metrics. In this paper, we show that these metrics have several drawbacks, making it challenging to understand the strengths and limitations of existing parsers. To address this, we propose a novel accuracy metric, called AML (Accuracy Metric for Log Parsing). AML is a robust accuracy metric that is inspired by research in the field of remote sensing. It is based on measuring omission and commission errors. We use AML to assess the accuracy of 14 log parsing tools applied to the parsing of 16 log datasets. We also show how AML compares to existing accuracy metrics. Our findings demonstrate that AML is a promising accuracy metric for log parsing compared to alternative solutions, which enables a comprehensive evaluation of log parsing tools to help better decision-making in selecting and improving log parsing techniques.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112154"},"PeriodicalIF":3.7,"publicationDate":"2024-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141705411","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Technical debt in AI-enabled systems: On the prevalence, severity, impact, and management strategies for code and architecture","authors":"Gilberto Recupito ,&nbsp;Fabiano Pecorelli ,&nbsp;Gemma Catolino ,&nbsp;Valentina Lenarduzzi ,&nbsp;Davide Taibi ,&nbsp;Dario Di Nucci ,&nbsp;Fabio Palomba","doi":"10.1016/j.jss.2024.112151","DOIUrl":"https://doi.org/10.1016/j.jss.2024.112151","url":null,"abstract":"<div><h3>Context:</h3><p>Artificial Intelligence (AI) is pervasive in several application domains and promises to be even more diffused in the next decades. Developing high-quality AI-enabled systems — software systems embedding one or multiple AI components, algorithms, and models — could introduce critical challenges for mitigating specific risks related to the systems’ quality. Such development alone is insufficient to fully address socio-technical consequences and the need for rapid adaptation to evolutionary changes. Recent work proposed the concept of AI technical debt, a potential liability concerned with developing AI-enabled systems whose impact can affect the overall systems’ quality. While the problem of AI technical debt is rapidly gaining the attention of the software engineering research community, scientific knowledge that contributes to understanding and managing the matter is still limited.</p></div><div><h3>Objective:</h3><p>In this paper, we leverage the expertise of practitioners to offer useful insights to the research community, aiming to enhance researchers’ awareness about the detection and mitigation of AI technical debt. Our ultimate goal is to empower practitioners by providing them with tools and methods. Additionally, our study sheds light on novel aspects that practitioners might not be fully acquainted with, contributing to a deeper understanding of the subject.</p></div><div><h3>Method:</h3><p>We develop a survey study featuring 53 AI practitioners, in which we collect information on the practical prevalence, severity, and impact of AI technical debt issues affecting the code and the architecture other than the strategies applied by practitioners to identify and mitigate them.</p></div><div><h3>Results:</h3><p>The key findings of the study reveal the multiple impacts that AI technical debt issues may have on the quality of AI-enabled systems (<em>e</em>.<em>g</em>., the high negative impact that <em>Undeclared consumers</em> has on security, whereas <em>Jumbled Model Architecture</em> can induce the code to be hard to maintain) and the little support practitioners have to deal with them, limited to apply manual effort for identification and refactoring.</p></div><div><h3>Conclusion:</h3><p>We conclude the article by distilling lessons learned and actionable insights for researchers.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112151"},"PeriodicalIF":3.7,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224001961/pdfft?md5=b02417a1294a463a8dd9b676949d1b4a&pid=1-s2.0-S0164121224001961-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141596840","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Temporal isolation assessment in virtualized safety-critical mixed-criticality systems: A case study on Xen hypervisor","authors":"Marcello Cinque,&nbsp;Luigi De Simone,&nbsp;Daniele Ottaviano","doi":"10.1016/j.jss.2024.112147","DOIUrl":"https://doi.org/10.1016/j.jss.2024.112147","url":null,"abstract":"<div><p>Today, we are witnessing the increasing use of the cloud and virtualization technologies, which are a prominent way for the industry to develop mixed-criticality systems (MCSs) and reduce <em>SWaP-C</em> factors (size, weight, power, and cost) by flexibly consolidating multiple critical and non-critical software on the same System-on-a-Chip (SoC). Unfortunately, using virtualization leads to several issues in assessing isolation aspects, especially temporal behaviors, which must be evaluated due to safety-related standards (e.g., EN50128 in the railway domain). This study proposes a systematic approach for verifying temporal isolation properties in virtualized MCSs to characterize and mitigate timing failures, which is a fundamental aspect of dependability. In particular, as proof of the effectiveness of our proposal, we exploited the real-time flavor of Xen hypervisor used to deploy a virtualized <em>2 out of 2</em>-based MCS scenario provided in the framework of an academic-industrial partnership, in the context of the railway domain. The results point out that virtualization overhead must be carefully tuned in a real industrial scenario according to the several features provided by a specific hypervisor solution. Further, we identify a set of directions toward employing virtualization in industry in the context of ARM-based mixed-criticality systems.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112147"},"PeriodicalIF":3.7,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224001924/pdfft?md5=f269cff9c3594f698621a5e15338501d&pid=1-s2.0-S0164121224001924-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141582685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"UVLHub: A feature model data repository using UVL and open science principles","authors":"David Romero-Organvidez ,&nbsp;José A. Galindo ,&nbsp;Chico Sundermann ,&nbsp;Jose-Miguel Horcas ,&nbsp;David Benavides","doi":"10.1016/j.jss.2024.112150","DOIUrl":"https://doi.org/10.1016/j.jss.2024.112150","url":null,"abstract":"<div><p>Feature models are the <em>de facto</em> standard for modelling variabilities and commonalities in features and relationships in software product lines. They are the base artefacts in many engineering activities, such as product configuration, derivation, or testing. Concrete models in different domains exist; however, many are in private or sparse repositories or belong to discontinued projects. The dispersion of knowledge of feature models hinders the study and reuse of these artefacts in different studies. The Universal Variability Language (UVL) is a community effort textual feature model language that promotes a common way of serializing feature models independently of concrete tools. Open science principles promote transparency, accessibility, and collaboration in scientific research. Although some attempts exist to promote feature model sharing, the existing solutions lack open science principles by design. In addition, existing and public feature models are described using formats not always supported by current tools. This paper presents <figure><img></figure> , a repository of feature models in UVL format. <figure><img></figure> provides a front end that facilitates the search, upload, storage, and management of feature model datasets, improving the capabilities of discontinued proposals. Furthermore, the tool communicates with Zenodo – one of the most well-known open science repositories – providing a permanent save of datasets and following open science principles. <figure><img></figure> includes existing datasets and is readily available to include new data and functionalities in the future. It is maintained by three active universities in variability modelling.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112150"},"PeriodicalIF":3.7,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S016412122400195X/pdfft?md5=2c04f3c6d777710e69a0d2ddb2012e6a&pid=1-s2.0-S016412122400195X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141596841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Project-specific code summarization with in-context learning","authors":"Shangbo Yun ,&nbsp;Shuhuai Lin ,&nbsp;Xiaodong Gu ,&nbsp;Beijun Shen","doi":"10.1016/j.jss.2024.112149","DOIUrl":"https://doi.org/10.1016/j.jss.2024.112149","url":null,"abstract":"<div><p>Automatically generating summaries for source code has emerged as a valuable task in software development. While state-of-the-art (SOTA) approaches have demonstrated significant efficacy in summarizing general code, they seldom concern code summarization for a specific project. Project-specific code summarization (PCS) poses special challenges due to the scarce availability of training data and the unique styles of different projects. In this paper, we empirically analyze the performance of Large Language Models (LLMs) on PCS tasks. Our study reveals that using appropriate prompts is an effective way to solicit LLMs for generating project-specific code summaries. Based on these findings, we propose a novel project-specific code summarization approach called P-CodeSum. P-CodeSum gathers a repository-level pool of (code, summary) examples to characterize the project-specific features. Then, it trains a neural prompt selector on a high-quality dataset crafted by LLMs using the example pool. The prompt selector offers relevant and high-quality prompts for LLMs to generate project-specific summaries. We evaluate against a variety of baseline approaches on six PCS datasets. Experimental results show that the P-CodeSum improves the performance by 5.9% (RLPG) to 101.51% (CodeBERT) on BLEU-4 compared to the state-of-the-art approaches in project-specific code summarization.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112149"},"PeriodicalIF":3.7,"publicationDate":"2024-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141583405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"BIT: A template-based approach to incremental and bidirectional model-to-text transformation","authors":"Xiao He ,&nbsp;Tao Zan","doi":"10.1016/j.jss.2024.112148","DOIUrl":"https://doi.org/10.1016/j.jss.2024.112148","url":null,"abstract":"<div><p>Model-driven development is a model-centric software development paradigm that automates the development process by converting high-level models into low-level code and documents. To maintain synchronization between models and code/documents — which can evolve independently — this paper introduces <span>BIT</span>, a bidirectional language that can serve as a conventional template language for model-to-text transformations. However, a <span>BIT</span> program can function as both a <em>printer</em>, generating text by filling template holes with values from the input model, and a <em>parser</em>, putting parsed values back into the model. <span>BIT</span> comprises a surface language for better usability and a core language for formal definition. We define the semantics of the core language based on the theory of bidirectional transformation, and provide the translation from the surface to the core. We present the proof sketch of the well behavedness of <span>BIT</span> as a formal evidence of soundness. We also conduct three case studies to empirically demonstrate the expressiveness and the effectiveness of <span>BIT</span>. Based on the proof and the case studies, <span>BIT</span> covers the major features of existing template languages, and offers sufficient expressiveness to define real-world model-to-text transformations that can be executed bidirectionally and incrementally.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112148"},"PeriodicalIF":3.7,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141605163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A fuzzy logic-based quality model for identifying microservices with low maintainability","authors":"Rahime Yılmaz ,&nbsp;Feza Buzluca","doi":"10.1016/j.jss.2024.112143","DOIUrl":"https://doi.org/10.1016/j.jss.2024.112143","url":null,"abstract":"<div><p>Microservice Architecture (MSA) is a popular architectural style that offers many advantages regarding quality attributes, including maintainability and scalability. Developing a system as a set of microservices with expected benefits requires a quality assessment strategy that is established on the measurements of the system's properties. This paper proposes a hierarchical quality model based on fuzzy logic to measure and evaluate the maintainability of MSAs considering ISO/IEC 250xy SQuaRE (System and Software Quality Requirements and Evaluation) standards. Since the qualitative bounds of low-level quality attributes are inherently ambiguous, we use a fuzzification technique to transform crisp values of code metrics into fuzzy levels and apply them as inputs to our quality model. The model generates fuzzy values for the quality sub-characteristics of the maintainability, i.e., modifiability and testability, converted to numerical values through defuzzification. In the last step, using the values of the sub-characteristics, we calculate numerical scores indicating the maintainability level of each microservice in the examined software system. This score was used to assess the quality of the microservices and decide whether they need refactoring. We evaluated our approach by creating a test set with the assistance of three developers, who reviewed and categorized the maintainability levels of the microservices in an open-source project based on their knowledge and experience. They labeled microservices as low, medium, or high, with low indicating the need for refactoring. Our method for identifying low-labeled microservices in the given test set achieved 94% accuracy, 78% precision, and 100% recall. These results indicate that our approach can assist designers in evaluating the maintainability quality of microservices.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"216 ","pages":"Article 112143"},"PeriodicalIF":3.7,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141582684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}