Rongcun Wang, Senlei Xu, Xingyu Ji, Yuan Tian, Lina Gong, Ke Wang
{"title":"An extensive study of the effects of different deep learning models on code vulnerability detection in Python code","authors":"Rongcun Wang, Senlei Xu, Xingyu Ji, Yuan Tian, Lina Gong, Ke Wang","doi":"10.1007/s10515-024-00413-4","DOIUrl":"10.1007/s10515-024-00413-4","url":null,"abstract":"<div><p>Deep learning has achieved great progress in automated code vulnerability detection. Several code vulnerability detection approaches based on deep learning have been proposed. However, few studies empirically studied the impacts of different deep learning models on code vulnerability detection in Python. For this reason, we strive to cover many more code representation learning models and classification models for vulnerability detection. We design and conduct an empirical study for evaluating the effects of the eighteen deep learning architectures derived from combinations of three representation learning models, i.e., Word2Vec, fastText, and CodeBERT, and six classification models, i.e., random forest, XGBoost, Multi-Layer Perception (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Gate Recurrent Unit (GRU) on code vulnerability detection in total. Additionally, two machine learning strategies i.e., the attention and bi-directional mechanisms are also empirically compared. The statistical significance and effect size analysis between different models are also conducted. In terms of <i>precision</i>, <i>recall</i>, and <i>F</i>-<i>score</i>, Word2Vec is better than Bidirectional Encoder Representations from Transformers CodeBERT and fastText. Likewise, long short-term memory (LSTM) and gated recurrent unit (GRU) are superior to other classification models we studied. The bi-directional LSTM and GRU with attention using Word2Vec are two optimal models for solving code vulnerability detection for Python code. Moreover, they have medium or large effect sizes on LSTM and GRU using only a single mechanism. Both the representation learning models and classification models have important influences on vulnerability detection in Python code. Likewise, the bi-directional and attention mechanisms can impact the performance of code vulnerability detection.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139657735","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaoning Shen, Chengbin Yao, Liyan Song, Jiyong Xu, Mingjian Mao
{"title":"Coevolutionary scheduling of dynamic software project considering the new skill learning","authors":"Xiaoning Shen, Chengbin Yao, Liyan Song, Jiyong Xu, Mingjian Mao","doi":"10.1007/s10515-023-00411-y","DOIUrl":"10.1007/s10515-023-00411-y","url":null,"abstract":"<div><p>In the process of software project development, completing tasks may require new skills that employees have not yet mastered due to factors such as requirement changes. However, existing studies on software project scheduling usually overlook such new skill demands. This paper designs the learning mechanism targeting the treatment of new skills for project employees, including how to select appropriate employees to learn new skills, the growth curves of new skill proficiencies and the adaptive dedication changes for the selected employees. Three common dynamic events are considered to establish a mathematical model for the dynamic software project scheduling problem considering the new skill learning. To solve the model, a multi-population coevolutionary algorithm-based predictive-reactive scheduling method is proposed in this paper. Three novel strategies are incorporated, which include a response mechanism to environmental changes, a population grouping strategy based on dual indicators, and a dynamic allocation of subpopulation size according to the variation trend of contribution. Systematic experimental results based on ten synthetic instances and three real-world instances show that when dynamic events occur, the proposed algorithm can quickly reschedule the tasks with a better duration, cost and stability compared with six state-of-the-art algorithms, helping project manager make a more informed decision.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":2.0,"publicationDate":"2024-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139509199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Marco Gerosa, Bianca Trinkenreich, Igor Steinmacher, Anita Sarma
{"title":"Can AI serve as a substitute for human subjects in software engineering research?","authors":"Marco Gerosa, Bianca Trinkenreich, Igor Steinmacher, Anita Sarma","doi":"10.1007/s10515-023-00409-6","DOIUrl":"10.1007/s10515-023-00409-6","url":null,"abstract":"<div><p>Research within sociotechnical domains, such as software engineering, fundamentally requires the human perspective. Nevertheless, traditional qualitative data collection methods suffer from difficulties in participant recruitment, scaling, and labor intensity. This vision paper proposes a novel approach to qualitative data collection in software engineering research by harnessing the capabilities of artificial intelligence (AI), especially large language models (LLMs) like ChatGPT and multimodal foundation models. We explore the potential of AI-generated synthetic text as an alternative source of qualitative data, discussing how LLMs can replicate human responses and behaviors in research settings. We discuss AI applications in emulating humans in interviews, focus groups, surveys, observational studies, and user evaluations. We discuss open problems and research opportunities to implement this vision. In the future, an integrated approach where both AI and human-generated data coexist will likely yield the most effective outcomes.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-01-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139424085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A security framework for mobile agent systems","authors":"Donies Samet, Farah Barika Ktata, Khaled Ghedira","doi":"10.1007/s10515-023-00408-7","DOIUrl":"10.1007/s10515-023-00408-7","url":null,"abstract":"<div><p>Security is a very important challenge in mobile agent systems due to the strong dependence of agents on the platform and vice versa. According to recent studies, most current mobile agent platforms suffer from significant limitations in terms of security when they face Denial of Service (DOS) attacks. Current security solutions even provided by the mobile agent platforms or by the literature focus essentially on individual attacks and are mainly based on static models that present a lack of the permissions definition and are not detailed enough to face collaborative DOS attacks executed by multiple agents or users. This paper presents a security framework that adds security defenses to mobile agent platforms. The proposed security framework implements a standard security model described using MA-UML (Mobile Agent-Unified Modeling Language) notations. The framework lets the administrator (of agents’ place) define a precise and fine-grained authorization policy to defend against DOS attacks. The authorization enforcement in the proposed framework is dynamic : the authorization decisions executed by the proposed framework are based upon run-time parameters like the amount of activity of an agent. We implement an experiment on a mobile agent system of e-marketplaces. Given that we focus essentially on the availability criterion, the performance of the proposed framework on a place is evaluated against DOS and DDOS attacks and investigated in terms of duration of execution that is the availability of the place.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139399877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Mining crowd sourcing repositories for open innovation in software engineering","authors":"Zeeshan Anwar, Hammad Afzal","doi":"10.1007/s10515-023-00410-z","DOIUrl":"10.1007/s10515-023-00410-z","url":null,"abstract":"<div><p>Various development tools have been introduced and the choice of suitable development tool depends on the particular context like the type of application to be developed, the development process and application domain, etc. The real challenge is to deliver new features at the right time with a faster development cycle. The selection of suitable development tools will help developers to save time and effort. In this research, we will explore software engineering repositories (like StackOverflow) to collect feedback from developers about development tools. This will explore which features in a development tool are most important, which features are missing, and which features require changes. The answers to these questions can be found by mining the community question-answering sites (CQA). We will use user feedback to innovate the new features in the development tool. Various techniques of Big Data, Data Mining, Deep Learning, and Transformers including Generative Pre-Training Transformer will be used in our research. Some of the major techniques include (i) data collection from CQA sites like StackOverflow, (ii) data preprocessing (iii) categories the data into various topics using topic modeling (iv) sentiment analysis of data to get positive or negative aspects of features (v) ranking of users and their feedback. The output of this research will categorize the users feedback into various ideas, this will help organizations to decide which features are required, which features are not required, which features are difficult or confusing, and which new features should be introduced into a new release.\u0000</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139399897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Correction to: A class integration test order generation approach based on Sarsa algorithm","authors":"Yun Li, Yanmei Zhang, Yanru Ding, Shujuan Jiang, Guan Yuan","doi":"10.1007/s10515-023-00412-x","DOIUrl":"10.1007/s10515-023-00412-x","url":null,"abstract":"","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2024-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139081477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Regression test selection in test-driven development","authors":"Zohreh Mafi, Seyed-Hassan Mirian-Hosseinabadi","doi":"10.1007/s10515-023-00405-w","DOIUrl":"10.1007/s10515-023-00405-w","url":null,"abstract":"<div><p>The large number of unit tests produced in the test-driven development (TDD) method and the iterative execution of these tests extend the regression test execution time in TDD. This study aims to reduce test execution time in TDD. We propose a TDD-based approach that creates traceable code elements and connects them to relevant test cases to support regression test selection during the TDD process. Our proposed hybrid technique combines text and syntax program differences to select related test cases using the nature of TDD. We use a change detection algorithm to detect program changes. Our experience is reported with a tool called RichTest, which implements this technique. In order to evaluate our work, seven TDD projects have been developed. The implementation results indicate that the RichTest plugin significantly decreases the number of test executions and also the time of regression testing despite considering the overhead time. The test suite effectively enables fault detection because the selected test cases are related to the modified partitions. Moreover, the test cases cover the entire modified partitions; accordingly, the selection algorithm is safe. The concept is particularly designed for the TDD method. Although this idea is applicable in any programming language, it is already implemented as a plugin in Java Eclipse.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139069288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Muneera Bano, Rashina Hoda, Didar Zowghi, Christoph Treude
{"title":"Large language models for qualitative research in software engineering: exploring opportunities and challenges","authors":"Muneera Bano, Rashina Hoda, Didar Zowghi, Christoph Treude","doi":"10.1007/s10515-023-00407-8","DOIUrl":"10.1007/s10515-023-00407-8","url":null,"abstract":"<div><p>The recent surge in the integration of Large Language Models (LLMs) like ChatGPT into qualitative research in software engineering, much like in other professional domains, demands a closer inspection. This vision paper seeks to explore the opportunities of using LLMs in qualitative research to address many of its legacy challenges as well as potential new concerns and pitfalls arising from the use of LLMs. We share our vision for the evolving role of the qualitative researcher in the age of LLMs and contemplate how they may utilize LLMs at various stages of their research experience.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2023-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138949097","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A class integration test order generation approach based on Sarsa algorithm","authors":"Yun Li, Yanmei Zhang, Yanru Ding, Shujuan Jiang, Guan Yuan","doi":"10.1007/s10515-023-00406-9","DOIUrl":"10.1007/s10515-023-00406-9","url":null,"abstract":"<div><p>Class integration test order generation is a key step in integration testing, researching this problem can help find unknown bugs and improve the efficiency of software testing. The challenge of this problem is ordering the classes to be integrated to minimize the cost of required stubs. However, the existing approaches of generating class integration test orders cannot satisfy this requirement well. Considering the excellent performance of reinforcement learning in sequence decision problems, this paper proposes a class integration test order generation approach based on Sarsa algorithm, which is a data-driven model-free reinforcement learning algorithm. This approach takes the stubbing complexity as the indicator to evaluate the stubbing cost and uses it to measure the quality of a class integration test order. The Sarsa algorithm is used to train the agent, and three indicators such as test return, dependency complexity, and the number of cycles are integrated into the design of the reward function to evaluate the merits of the current action. By recording an action path of the agent from its initial state to its termination state, a class integration test order can be obtained. The experimental results on 10 systems show that the class integration test order generation approach based on Sarsa algorithm can generate the class integration test orders with lower stubbing cost.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138581507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xingzi Yu, Wei Tang, Tianlei Xiong, Wengang Chen, Jie He, Bin Yang, Zhengwei Qi
{"title":"Enhancing embedded systems development with TS(^-)","authors":"Xingzi Yu, Wei Tang, Tianlei Xiong, Wengang Chen, Jie He, Bin Yang, Zhengwei Qi","doi":"10.1007/s10515-023-00404-x","DOIUrl":"10.1007/s10515-023-00404-x","url":null,"abstract":"<div><p>The lack of flexibility and safety in C language development has been criticized for a long time, causing detriments to the development cycle and software quality in the embedded systems domain. TypeScript, as an optionally-typed dynamic language, offers the flexibility and safety that developers desire. With the advancement of Ahead-of-Time (AOT) compilation technologies for TypeScript and JavaScript, it has become feasible to write embedded applications using TypeScript. Despite the availability of writing AOT compiled programs with TypeScript, implementing a compiler toolchain for this purpose requires substantial effort. To simplify the design of languages and compilers, this paper presents a new compiler toolchain design methodology called TS<span>(^-)</span>, which advocates the generation of target intermediate language code (such as C) from TypeScript rather than the construction of higher-level compiler tools and type systems on top of the intermediate language. TS<span>(^-)</span> not only simplifies the design of the system but also provides developers with a quasi-native TypeScript development experience. This paper also presents <span>Ts2Wasm</span>, a prototype that implements TS<span>(^-)</span> and allows compiling a language subset of TypeScript to WebAssembly (WASM). The tests in the TypeScript repository show that <span>Ts2Wasm</span> provides 3.8× as many features compared to the intermediate language (AssemblyScript). Regarding performance, <span>Ts2Wasm</span> offers a significant speed-up of 1.4× to 19×. Meanwhile, it imposes over 65% less memory overhead compared to Node.js in most cases.</p></div>","PeriodicalId":55414,"journal":{"name":"Automated Software Engineering","volume":"31 1","pages":""},"PeriodicalIF":3.4,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138507038","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}