{"title":"Focus on New Test Cases in Continuous Integration Testing based on Reinforcement Learning","authors":"Fanliang Chen, Zheng Li, Y. Shang, Yang Yang","doi":"10.1109/QRS57517.2022.00088","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00088","url":null,"abstract":"In software regression testing, newly added test cases are more likely to fail, and therefore, should be prioritized for execution. In software regression testing for continuous integration, reinforcement learning-based approaches are promising and the RETECS (Reinforced Test Case Prioritization and Selection) framework is a successful application case. RETECS uses an agent composed of a neural network to predict the priority of test cases, and the agent needs to learn from historical information to make improvements. However, the newly added test cases have no historical execution information, thus using RETECS to predict their priority is more like ‘random’. In this paper, we focus on new test cases for continuous integration testing, and on the basis of the RETECS framework, we first propose a priority assignment method for new test cases to ensure that they can be executed first. Secondly, continuous integration is a fast iterative integration method where new test cases have strong fault detection capability within the latest periods. Therefore, we further propose an additional reward method for new test cases. Finally, based on the full lifecycle management, the ‘new’ additional rewards need to be terminated within a certain period, and this paper implements an empirical study. We conducted 30 iterations of the experiment on 12 datasets and our best results were 19.24%, 10.67%, and 34.05 positions better compared to the best parameter combination in RETECS for the NAPFD (Normalized Average Percentage of Faults Detected), RECALL and TTF (Test to Fail) metrics, respectively.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116873229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaojuan Wang, Wenyu Zhang, Shanyan Lai, Chunyang Ye, Hui Zhou
{"title":"The Use of Pretrained Model for Matching App Reviews and Bug Reports","authors":"Xiaojuan Wang, Wenyu Zhang, Shanyan Lai, Chunyang Ye, Hui Zhou","doi":"10.1109/QRS57517.2022.00034","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00034","url":null,"abstract":"Matching APP reviews with bug reports can help APP developers to quickly identify new bugs from the users’ feedback. Existing solutions represent the semantics of APP reviews and bug reports via carefully designed features and models, the performance of which however depends heavily on the manually designed model and the training data set. Large-scale pretrained models can well capture the semantics of text and have demonstrated their success in many NLP tasks. Inspired by this, we explore the effect of various pretrained models on the matching accuracy of app review and bug report. We conduct a systematic study to analyze the factors of four major pretrained models (including T5, Sentence T5, Sentence MiniLM, Sentence BERT and so on) on the matching accuracy. We find that the accuracy of Sentence T5 and Sentence MiniLM in four open source applications is significantly greater than that of the state-of-the-art approach DeepMatcher. Based on the findings, we design a novel approach to match the APP reviews with bug reports based on the pretrained model Sentence T5 and Sentence MiniLM to calculate the sentence similarity. We test it on four open source applications and the results show that our method outperforms the existing solution. On average, the precision of Sentence T5 and Sentence MiniLM are increased by 17% and 13%, respectively, and the hit ratio are increased by 15% and 14%, respectively.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134104201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visualization-Based Software Defect Prediction via Convolutional Neural Network with Global Self-Attention","authors":"Shaojian Qiu, Shaosheng Wang, Xuhong Tian, Mengyang Huang, Qiong Huang","doi":"10.1109/QRS57517.2022.00029","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00029","url":null,"abstract":"Defect prediction technology helps software quality assurance teams understand the distribution of software defects, which can assist them to allocate testing and verification resources appropriately. Current visualization-based software defect prediction methods lack spatial and global information of code images during the feature extraction process. To solve the problem of incomplete information, this paper proposes a Convolutional Neural Network with Global Self-Attention (CNN-GSA). The method converts codes into corresponding images and uses an improved convolutional neural network, which combines channel attention, spatial attention, and self-attention mechanisms in a global attention layer, to extract defect-related structural and semantic features in code images. Empirical study shows that the model built with the features generated by CNN-GSA can achieve better F-measure results in defect prediction tasks.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132855200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automatic Collaborative Testing of Applications Integrating Text Features and Priority Experience Replay","authors":"Lizhi Cai, Jin Wang, Mingang Chen, Jilong Wang","doi":"10.1109/QRS57517.2022.00020","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00020","url":null,"abstract":"With the popularity of deep reinforcement learning(DRL), people have great interest in using deep reinforcement learning for application automated testing. However, most automated testing methods based on reinforcement learning ignore text information, use random sampling in experience replay and ignore the characteristics of Android automated testing. To solve above problem, this paper proposes ITPRTesting(Integrated Text feature information and Priority experience in Testing). It extracts the text information in the interface and uses the BERT algorithm to generate sentence vectors. It fuses the interactive control feature diagram(ICFD), which is mentioned in the previous work, and text information as the state required by reinforcement learning. And in reinforcement learning, the priority experience replay is combined, also the traditional priority experience replay is improved. This paper has carried out experiments on 10 open source applications. The experimental results show that ITPRTesting is superior to other methods in statement coverage and branch coverage.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134312479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"QRS 2022 Program Committee","authors":"","doi":"10.1109/qrs57517.2022.00008","DOIUrl":"https://doi.org/10.1109/qrs57517.2022.00008","url":null,"abstract":"","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120841075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"API Misuse Detection Method Based on Transformer","authors":"Jingbo Yang, Jian Ren, Wenjun Wu","doi":"10.1109/QRS57517.2022.00100","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00100","url":null,"abstract":"Software developers need to take advantage of a variety of APIs (application programming interface) in their programs to implement specific functions. The problem of API misuses often arises when developers have incorrect understandings about the new APIs without carefully reading API documents. In order to avoid software defects caused by API misuse, researchers have explored multiple methods, including using AI(artificial intelligence) technology.As a kind of neural network in AI, Transformer has a good sequence processing ability, and the self attention mechanism used by Transformer can better catch the relation in a sequence or between different sequences. Besides it has a good model interpretability. From the perspective of combining API misuse detection with AI, this paper implements a standard Transformer model and a target-combination Transformer model to the learning of API usage information in a named API call sequence extracted from API usage program code. Then we present in the paper the way that our models use API usage information to detect if an API is misused in code. We use F1, precision and recall to evaluate the detection ability and show the advantages of our models in these three indexes. Besides, our models based on Transformer both have a better convergence. Finally, this paper explains why the models based on Transformer has a better performance by showing attention weight among different elements in code.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115967574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"CRUST: Towards a Unified Cross-Language Program Analysis Framework for Rust","authors":"Shuang Hu, Baojian Hua, Lei Xia, Yang Wang","doi":"10.1109/QRS57517.2022.00101","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00101","url":null,"abstract":"Rust is a new safe system programming language enforcing safety guarantees by novel language features, a rich type system, and strict compile-time checking rules, and thus has been used extensively to build system software. For multilingual Rust applications containing external C code, memory security vulnerabilities can occur due to the intrinsically unsafe nature of C and the improper interactions between Rust and C. Unfortunately, existing security studies on Rust only focus on pure Rust code but cannot analyze either the native C code or the Rust/C interactions in multilingual Rust applications. As a result, the lack of such studies may defeat the guarantee that Rust is a safe language.This paper presents CRust, a unified program analysis framework across Rust and C, which enables program analyses to understand the semantics of C code by translating Rust and C into a unified specification language. The CRust framework consists of three key components: (1) a unified specification language CRustIR, which is a strong-typed low-level intermediate language suitable for program analysis; (2) a transformation to build models of C code by converting C code into CRustIR; and (3) program analysis algorithms on CRustIR to detect security vulnerabilities. We have implemented a software prototype for CRust, and have conducted extensive experiments to evaluate its effectiveness and performance. Experimental results demonstrated that CRust can effectively detect common memory security vulnerabilities caused by the interaction of Rust and C that are missed by state-of-the-art tools. In addition, CRust is efficient in bringing negligible overhead (0.23 seconds on average).","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116189126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluation of the Architecture Alternatives for Real-Time Intrusion Detection Systems for Vehicles","authors":"Mubark Jedh, Jian Kai Lee, L. B. Othmane","doi":"10.1109/QRS57517.2022.00091","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00091","url":null,"abstract":"Attackers demonstrated the use of remote access to the in-vehicle network of connected vehicles to take control of these vehicles. Machine-learning-based Intrusion Detection Systems (IDSs) techniques have been proposed for the detection of such attacks. The evaluations of some of these IDSs showed their efficacy in terms of accuracy in detecting message injections but were performed offline, which limits the confidence in their use for real-time protection scenarios. This paper evaluates four architecture designs for real-time IDS for connected vehicles using Controller Area Network (CAN) datasets collected from a moving vehicle under malicious speed reading message injections. The evaluation shows that a real-time IDS for a connected vehicle designed as a separate process for CAN Bus monitoring and another one for anomaly detection engine is reliable (does not lose messages) and could be used for real-time resilience mechanisms as a response to cyber-attacks.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122033817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thorn Jansen, Fernando Pastor Ricós, Yaping Luo, K. Vlist, R. V. Dalen, Pekka Aho, T. Vos
{"title":"Scriptless GUI Testing on Mobile Applications","authors":"Thorn Jansen, Fernando Pastor Ricós, Yaping Luo, K. Vlist, R. V. Dalen, Pekka Aho, T. Vos","doi":"10.1109/QRS57517.2022.00113","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00113","url":null,"abstract":"Traditionally, end-to-end testing of mobile apps is either performed manually or automated with test scripts. However, manual GUI testing is expensive and slow, and test scripts are fragile for GUI changes, resulting in high maintenance costs. Scriptless testing attempts to address the costs associated with GUI testing. Existing scriptless approaches for mobile testing do not seem to fit the requirements of the industry, specifically those of the ING. This study presents an extension to open source TESTAR tool to support scriptless GUI testing of Android and iOS applications. We present an initial validation of the tool on an industrial setting at the ING. From the validation, we determine that the extended TESTAR outperforms two other state-of-the-art scriptless testing tools for Android in terms of code coverage, and achieves similar performance as the scripted test automation already in use at the ING. Moreover, we see that the scriptless approach covers parts of the application under test that the existing test scripts did not cover, showing the complementarity of the approaches, providing more value for the testers.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122360722","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}