Journal of Systems and Software最新文献

筛选
英文 中文
Balancing quality and efficiency: An improved non-autoregressive model for pseudocode-to-code conversion 平衡质量与效率:伪代码到代码转换的改进型非自回归模型
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-09-12 DOI: 10.1016/j.jss.2024.112206
Yongrui Xu, Shang Zheng, Haitao Zou, Hualong Yu, Shang Gao
{"title":"Balancing quality and efficiency: An improved non-autoregressive model for pseudocode-to-code conversion","authors":"Yongrui Xu,&nbsp;Shang Zheng,&nbsp;Haitao Zou,&nbsp;Hualong Yu,&nbsp;Shang Gao","doi":"10.1016/j.jss.2024.112206","DOIUrl":"10.1016/j.jss.2024.112206","url":null,"abstract":"<div><p>Pseudocode can efficiently represent algorithm logic, but manual conversion to executable code requires more time. Recent works have applied autoregressive (AR) models to automate pseudocode-to-code conversion, achieving good results but slow generation speed. Non-autoregressive (NAR) models offer the advantage of parallel generation. However, they face challenges in effectively capturing contextual information, leading to a potential degradation in the quality of the generated output. This paper presents an improved NAR model for balancing quality and efficiency in pseudocode conversion. Firstly, two strategies are proposed to address out-of-vocabulary and repetition problems. Secondly, an improved NAR model is built using linear smoothing and adaptive techniques in the transition matrix, which can mitigate the “<em>winner takes all</em>” effect. Finally, a new synthesis potential metric is proposed for evaluating pseudocode conversion. Experimental results show that the proposed method matches AR model performance while accelerating generation over 10-fold. Further, the proposed NAR model reduces the gap with the AR model in terms of the BLEU score on the EN-DE and DE-EN tasks of the WMT14 machine translation.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112206"},"PeriodicalIF":3.7,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142172727","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An intelligent test management system for optimizing decision making during software testing 优化软件测试决策的智能测试管理系统
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-09-06 DOI: 10.1016/j.jss.2024.112202
Albin Lönnfält , Viktor Tu , Gregory Gay , Animesh Singh , Sahar Tahvili
{"title":"An intelligent test management system for optimizing decision making during software testing","authors":"Albin Lönnfält ,&nbsp;Viktor Tu ,&nbsp;Gregory Gay ,&nbsp;Animesh Singh ,&nbsp;Sahar Tahvili","doi":"10.1016/j.jss.2024.112202","DOIUrl":"10.1016/j.jss.2024.112202","url":null,"abstract":"<div><p>To ensure the proper testing of any software product, it is imperative to cover various functional and non-functional requirements at different testing levels (e.g., unit or integration testing). Ensuring appropriate testing requires making a series of decisions—e.g., assigning features to distinct Continuous Integration (CI) configurations or determining which test specifications to automate. Such decisions are generally made manually and require in-depth domain knowledge. This study introduces, implements, and evaluates ITMOS (Intelligent Test Management Optimization System), an intelligent test management system designed to optimize decision-making during the software testing process. ITMOS efficiently processes new requirements presented in natural language, segregating each requirement into appropriate CI configurations based on predefined quality criteria. Additionally, ITMOS has the capability to suggest a set of test specifications for test automation. The feasibility and potential applicability of the proposed solution were empirically evaluated in an industrial telecommunications project at Ericsson. In this context, ITMOS achieved accurate results for decision-making tasks, exceeding the requirements set by domain experts.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112202"},"PeriodicalIF":3.7,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142161757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Feature transformation for improved software bug detection and commit classification 改进软件错误检测和提交分类的特征转换
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-09-06 DOI: 10.1016/j.jss.2024.112205
Sakib Mostafa, Shamse Tasnim Cynthia, Banani Roy, Debajyoti Mondal
{"title":"Feature transformation for improved software bug detection and commit classification","authors":"Sakib Mostafa,&nbsp;Shamse Tasnim Cynthia,&nbsp;Banani Roy,&nbsp;Debajyoti Mondal","doi":"10.1016/j.jss.2024.112205","DOIUrl":"10.1016/j.jss.2024.112205","url":null,"abstract":"<div><p>Testing and debugging software to fix bugs is considered one of the most important stages of the software life cycle. Many studies have investigated ways to predict bugs in software artifacts using machine learning techniques. It is important to consider the explanatory aspects of such models for reliable prediction. In this paper, we show how feature transformation can significantly improve prediction accuracy and provide insight into the inner workings of bug prediction models. We propose a new approach for bug prediction that first extracts the features, then finds a weighted transformation of these features using a genetic algorithm that best separates bugs from non-bugs when plotted in a low-dimensional space, and finally, trains predictive models using the transformed dataset. In our experiment using the proposed feature transformation, the traditional machine learning and deep learning classifiers achieved an average improvement of 4.25% and 9.6% in recall values for bug classification over 8 software systems compared to the models built on original data. We also examined the generalizability of our concept for multiclass classification tasks such as commit classification in software systems and found modest improvements in F1-scores (sometimes up to 3%) for traditional machine learning models and 4% with deep learning models.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112205"},"PeriodicalIF":3.7,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224002498/pdfft?md5=24be736d13c3422f3ae6248d88baf8da&pid=1-s2.0-S0164121224002498-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142161759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Attributes of a great requirements engineer 优秀需求工程师的特质
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-09-05 DOI: 10.1016/j.jss.2024.112200
Larissa Barbosa , Sávio Freire , Rita S.P. Maciel , Manoel Mendonça , Marcos Kalinowski , Zadia Codabux , Rodrigo Spínola
{"title":"Attributes of a great requirements engineer","authors":"Larissa Barbosa ,&nbsp;Sávio Freire ,&nbsp;Rita S.P. Maciel ,&nbsp;Manoel Mendonça ,&nbsp;Marcos Kalinowski ,&nbsp;Zadia Codabux ,&nbsp;Rodrigo Spínola","doi":"10.1016/j.jss.2024.112200","DOIUrl":"10.1016/j.jss.2024.112200","url":null,"abstract":"<div><h3>Context and motivation:</h3><p>Several studies have investigated attributes of great software practitioners. However, the investigation of such attributes is still missing in Requirements Engineering (RE). The current knowledge on attributes of great software practitioners might not be easily translated to the context of RE because its activities are, usually, less technical and more human-centered than other software engineering activities.</p></div><div><h3>Question/problem:</h3><p>This work aims to investigate which are the attributes of great requirements engineers, the relationship between them, and strategies that can be employed to obtain these attributes. We follow a method composed of a survey with 18 practitioners and follow up interviews with 11 of them.</p></div><div><h3>Principal ideas/results:</h3><p><em>Investigative ability in talking to stakeholders</em>, <em>judicious</em>, and <em>understand the business</em> are the most commonly mentioned attributes amongst the set of 22 attributes identified, which were grouped into four categories. We also found 38 strategies to improve RE skills. Examples are <em>training</em>, <em>talking to all stakeholders</em>, and <em>acquiring domain knowledge</em>.</p></div><div><h3>Contribution:</h3><p>The attributes, their categories, and relationships are organized into a map. The relations between attributes and strategies are represented in a Sankey diagram. Software practitioners can use our findings to improve their understanding about the role and responsibilities of requirements engineers.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112200"},"PeriodicalIF":3.7,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142167497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Automatic generation of efficient oracles: The less-than case 自动生成高效报文小于的情况
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-09-04 DOI: 10.1016/j.jss.2024.112203
Javier Sanchez-Rivero , Daniel Talaván , Jose Garcia-Alonso , Antonio Ruiz-Cortés , Juan Manuel Murillo
{"title":"Automatic generation of efficient oracles: The less-than case","authors":"Javier Sanchez-Rivero ,&nbsp;Daniel Talaván ,&nbsp;Jose Garcia-Alonso ,&nbsp;Antonio Ruiz-Cortés ,&nbsp;Juan Manuel Murillo","doi":"10.1016/j.jss.2024.112203","DOIUrl":"10.1016/j.jss.2024.112203","url":null,"abstract":"<div><p>Grover’s algorithm is a well-known contribution to quantum computing. It searches one value within an unordered sequence faster than any classical algorithm. A fundamental part of this algorithm is the so-called oracle, a quantum circuit that marks the quantum state corresponding to the desired value. A generalisation of it is the oracle for Amplitude Amplification, that marks multiple desired states. In this work we present a classical algorithm that builds a phase-marking oracle for Amplitude Amplification. This oracle performs a less-than operation, marking states representing natural numbers smaller than a given one. Results of both simulations and experiments are shown to prove its functionality. This less-than oracle implementation works on any number of qubits and does not require any ancilla qubits. Regarding depth, the proposed implementation is compared with the one generated by Qiskit automatic method, <em>Diagonal</em>. We show that the depth of our less-than oracle implementation is always lower. In addition, a comparison with another method for oracle generation in terms of gate count is also conducted to prove the efficiency of our method. The result presented here is part of a research work that aims to achieve reusable quantum operations that can be composed to perform more complex ones. The final aim is to provide Quantum Developers with tools that can be easily integrated in their programs/circuits.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112203"},"PeriodicalIF":3.7,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224002474/pdfft?md5=045c01aa1d649973c047ead28c3395fc&pid=1-s2.0-S0164121224002474-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142161758","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Exploring the problems, their causes and solutions of AI pair programming: A study on GitHub and Stack Overflow 探索人工智能结对编程的问题、原因和解决方案:对 GitHub 和 Stack Overflow 的研究
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-09-02 DOI: 10.1016/j.jss.2024.112204
Xiyu Zhou , Peng Liang , Beiqi Zhang , Zengyang Li , Aakash Ahmad , Mojtaba Shahin , Muhammad Waseem
{"title":"Exploring the problems, their causes and solutions of AI pair programming: A study on GitHub and Stack Overflow","authors":"Xiyu Zhou ,&nbsp;Peng Liang ,&nbsp;Beiqi Zhang ,&nbsp;Zengyang Li ,&nbsp;Aakash Ahmad ,&nbsp;Mojtaba Shahin ,&nbsp;Muhammad Waseem","doi":"10.1016/j.jss.2024.112204","DOIUrl":"10.1016/j.jss.2024.112204","url":null,"abstract":"<div><p>With the recent advancement of Artificial Intelligence (AI) and Large Language Models (LLMs), AI-based code generation tools become a practical solution for software development. GitHub Copilot, the AI pair programmer, utilizes machine learning models trained on a large corpus of code snippets to generate code suggestions using natural language processing. Despite its popularity in software development, there is limited empirical evidence on the actual experiences of practitioners who work with Copilot. To this end, we conducted an empirical study to understand the problems that practitioners face when using Copilot, as well as their underlying causes and potential solutions. We collected data from 473 GitHub issues, 706 GitHub discussions, and 142 Stack Overflow posts. Our results reveal that (1) <em>Operation Issue</em> and <em>Compatibility Issue</em> are the most common problems faced by Copilot users, (2) <em>Copilot Internal Error</em>, <em>Network Connection Error</em>, and <em>Editor/IDE Compatibility Issue</em> are identified as the most frequent causes, and (3) <em>Bug Fixed by Copilot</em>, <em>Modify Configuration/Setting</em>, and <em>Use Suitable Version</em> are the predominant solutions. Based on the results, we discuss the potential areas of Copilot for enhancement, and provide the implications for the Copilot users, the Copilot team, and researchers.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112204"},"PeriodicalIF":3.7,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142150765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
DeepFeature: Guiding adversarial testing for deep neural network systems using robust features DeepFeature:利用鲁棒特征指导深度神经网络系统的对抗测试
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-08-31 DOI: 10.1016/j.jss.2024.112201
Lichao Feng , Xingya Wang , Shiyu Zhang , Zhihong Zhao
{"title":"DeepFeature: Guiding adversarial testing for deep neural network systems using robust features","authors":"Lichao Feng ,&nbsp;Xingya Wang ,&nbsp;Shiyu Zhang ,&nbsp;Zhihong Zhao","doi":"10.1016/j.jss.2024.112201","DOIUrl":"10.1016/j.jss.2024.112201","url":null,"abstract":"<div><p>With the deployment of Deep Neural Network (DNN) systems in security-critical fields, more and more researchers are concerned about DNN robustness. Unfortunately, DNNs are vulnerable to adversarial attacks and produce completely wrong outputs. This inspired numerous testing works devoted to improving the adversarial robustness of DNNs. Coverage and uncertainty criteria were proposed to guide sample selections for DNN retraining. However, they are greatly limited to evaluating DNN abnormal behaviors rather than locating the root cause of adversarial vulnerability. This work aims to bridge this gap. We propose an adversarial testing framework, DeepFeature, using robust features. DeepFeature generates robust features related to the model decision-making. It locates the weak features within these features that fail to be transformed by the DNN. They are the main culprits of vulnerability. DeepFeature selects diverse samples containing weak features for adversarial retraining. Our evaluation shows that DeepFeature significantly improves overall robustness, average improved by 77.83%, and individual robustness, average improved by 42.81‰, of the models in adversarial testing. Compared with coverage and uncertainty criteria, these two performances are improved by 3.93% and 15.00% in DeepFeature, respectively. The positive correlation coefficient between DeepFeature and improved robustness can achieve 0.858, and the <span><math><mi>p</mi></math></span>-value is 0.001.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112201"},"PeriodicalIF":3.7,"publicationDate":"2024-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142167885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
How are discussions linked? A link analysis study on GitHub Discussions 讨论是如何链接的?GitHub 讨论的链接分析研究
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-08-30 DOI: 10.1016/j.jss.2024.112196
Márcia Lima , Igor Steinmacher , Denae Ford , Grace Vorreuter , Ludimila Gonçalves , Tayana Conte , Bruno Gadelha
{"title":"How are discussions linked? A link analysis study on GitHub Discussions","authors":"Márcia Lima ,&nbsp;Igor Steinmacher ,&nbsp;Denae Ford ,&nbsp;Grace Vorreuter ,&nbsp;Ludimila Gonçalves ,&nbsp;Tayana Conte ,&nbsp;Bruno Gadelha","doi":"10.1016/j.jss.2024.112196","DOIUrl":"10.1016/j.jss.2024.112196","url":null,"abstract":"<div><p>Software development requires collaborative efforts and consensus among developers, emphasizing the need for effective communication and knowledge sharing within the teams. In line with this, GitHub introduced GitHub Discussions, a collaborative communication feature for the community around an open-source or internal project. The user-friendly interface of Discussions facilitates easy sharing of links to different resources. Maintainers highlight this feature’s significance, enabling users to maintain a well-informed project environment. We hypothesize that link-sharing activities in Discussions contribute to disseminating project knowledge. To investigate this hypothesis, we conducted a mixed-method study combining qualitative and quantitative analysis based on a convenience sample of ten open-source projects. We aimed to gain insight into the scope and intentions behind these shared links. Our findings indicate that link-sharing activities are common in the Discussions. Users share links to resources directly or indirectly related to the project/repository. Discussions users share links to project documentation, source code, and issues, aiming to clarify concepts, provide supplementary information, offer context to questions, and suggest new features. These findings offer insights for project maintainers to understand Discussions usage better and enable the GitHub Engineering team to promote the feature adoption.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"219 ","pages":"Article 112196"},"PeriodicalIF":3.7,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142150764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Software modernization powered by dynamic language product lines 动态语言产品线推动软件现代化
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-08-28 DOI: 10.1016/j.jss.2024.112188
Walter Cazzola, Luca Favalli
{"title":"Software modernization powered by dynamic language product lines","authors":"Walter Cazzola,&nbsp;Luca Favalli","doi":"10.1016/j.jss.2024.112188","DOIUrl":"10.1016/j.jss.2024.112188","url":null,"abstract":"<div><p>Legacy software poses a critical challenge for organizations due to the costs of maintaining and modernizing outdated systems, as well as the scarcity of experts in aging programming languages. The issue extends beyond commercial applications, affecting public administration, as exemplified by the urgent need for <span>COBOL</span> programmers during the COVID-19 pandemic. In response, this work introduces a modernization approach based on dynamic language product lines, a subset of dynamic software product lines. This approach leverages open language implementations and dynamically generated micro-languages for the incremental migration of legacy systems to modern technologies. The language can be reconfigured at runtime to adapt to the execution of either legacy or modern code, and to generate a compatibility layer between the data types handled by the two languages. Through this process, the costs of modernizing legacy systems can be spread across several iterations, as developers can replace legacy code incrementally, with legacy and modern code coexisting until a complete refactoring is possible. By moving the overhead of making legacy and modern features work together in a hybrid system from the system implementation to the language implementation, the quality of the system itself does not degrade due to the introduction of glue code. To demonstrate the practical applicability of this approach, we present a case study on a <span>COBOL</span> system migration to <span>Java</span>. Using the <span>Neverlang</span> language workbench to create modular and reconfigurable language implementations, both the <span>COBOL</span> interpreter and the application evolve to spread the development effort across several iterations. Through this study, this work presents a viable solution for organizations dealing with the complexity of modernizing legacy software to contemporary technologies. The contributions of this work are (i) a language-oriented, incremental refactoring process for legacy systems, (ii) a concrete application of open language implementations, and (iii) a general template for the implementation of interoperability between languages in hybrid systems.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"218 ","pages":"Article 112188"},"PeriodicalIF":3.7,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224002322/pdfft?md5=087cacb89bcf9dc4a2cefae984eef08a&pid=1-s2.0-S0164121224002322-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Research artifacts for human-oriented experiments in software engineering: An ACM badges-driven structure proposal 软件工程中以人为导向的实验研究工件:ACM 徽章驱动结构提案
IF 3.7 2区 计算机科学
Journal of Systems and Software Pub Date : 2024-08-27 DOI: 10.1016/j.jss.2024.112187
Cathy Guevara-Vega , Beatriz Bernárdez , Margarita Cruz , Amador Durán , Antonio Ruiz-Cortés , Martin Solari
{"title":"Research artifacts for human-oriented experiments in software engineering: An ACM badges-driven structure proposal","authors":"Cathy Guevara-Vega ,&nbsp;Beatriz Bernárdez ,&nbsp;Margarita Cruz ,&nbsp;Amador Durán ,&nbsp;Antonio Ruiz-Cortés ,&nbsp;Martin Solari","doi":"10.1016/j.jss.2024.112187","DOIUrl":"10.1016/j.jss.2024.112187","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Context:&lt;/h3&gt;&lt;p&gt;The Open Science (OS) movement promotes the value of making public the research artifacts (datasets, analysis scripts, guidelines, etc.) used during empirical studies. OS is widely known in areas such as Medicine or Biology, where the process of sharing research artifacts is subject to strict protocols. Unfortunately, in Software Engineering (SE), this process is carried out in a non-systematic way, resulting in incomplete or inaccurate material shared by researchers, which hinders the reproducibility and replicability of empirical studies. Nevertheless, in recent years, it seems that the Empirical Software Engineering (ESE) community is embracing some of the proposed OS initiatives, such as the one proposed by the Association for Computing Machinery (ACM), which provides a badge system to evaluate the quality of a research artifact. This badge system has been adopted by several SE conferences as a method of assessing research artifacts.&lt;/p&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Aims:&lt;/h3&gt;&lt;p&gt;Focusing on human-oriented experiments (HOEs) in SE, whose research artifacts are more complex than those for computational experiments, this work applies Design Science Research (DSR) with a twofold purpose: (i) review the current status of HOEs research artifacts publication through evaluation of this practice in the most relevant ESE journals , and (ii) propose a structured outline for HOEs research artifacts driven by the aforementioned ACM badging policy.&lt;/p&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Method:&lt;/h3&gt;&lt;p&gt;Regarding the first purpose, we carried out a survey to analyze the current status of the publication of research artifacts considering relevant peer review journals and the quality of 106 research artifacts published in these journals with respect to the ACM badging policy. For the second purpose, an iterative process was carried out to review the content of 106 research artifacts research and their concordance with ACM badges, obtaining a structured scheme for HOEs research artifacts that has been validated through a detailed review of 12 research artifacts obtained from some of those of ACM badges in relevant SE conferences. In addition, we validated the proposal in the research artifacts of 2 of our own experiments.&lt;/p&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results:&lt;/h3&gt;&lt;p&gt;Our survey reveals issues such as the 39,70% of journal studies making completely accessible their research artifacts; most of the analyzed research artifacts are incomplete; the most common repositories used in the ESE community to share the research artifacts are GitHub, institutional repositories, and Zenodo. On the other hand, the validated and structured research artifact outline consists of a list of ordered sections containing a set of artifacts, which can be mandatory or not to achieve a particular ACM badge. For its internal validation, several improvement iterations on the first release of the outline have been carried out based on the conference guidelines, the ACM badging policy, and other relevant proposals.","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":"218 ","pages":"Article 112187"},"PeriodicalIF":3.7,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0164121224002310/pdfft?md5=fb4a53f470e5e9d69349ac3f01883dd0&pid=1-s2.0-S0164121224002310-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142087631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信