{"title":"A Survey of the Use of Test Report in Crowdsourced Testing","authors":"Song Huang, Hao Chen, Zhan-wei Hui, Yuchan Liu","doi":"10.1109/QRS51102.2020.00062","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00062","url":null,"abstract":"With the rise of crowdsourced software testing in recent years, the issuers of crowd test tasks can usually collect a large number of test reports after the end of the task. These reports have insufficient validity and completeness, and manual review often takes a lot of time and effort. The crowdsourced test task publisher hopes that after the crowdsourced platform collects the test report, it can analyze the validity and completeness of the report to determine the severity of the report and improve the efficiency of crowdsourced software testing. In the past ten years, researchers have used various technologies (such as natural language processing, information retrieval, machine learning, deep learning) to assist in analyzing reports to improve the efficiency of report review. We have summarized the relevant literature of report analysis in the past ten years, and then classified from report classification, duplicate report detection, report prioritization, report refactoring, and summarized the most important research work in each area. Finally, we propose research trends in these areas and analyze the challenges and opportunities facing crowdsourced test report analysis.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134084448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Max Landauer, Florian Skopik, Markus Wurzenberger, Wolfgang Hotwagner, A. Rauber
{"title":"Have It Your Way: Generating Customized Log Data Sets with a Model-driven Simulation Testbed","authors":"Max Landauer, Florian Skopik, Markus Wurzenberger, Wolfgang Hotwagner, A. Rauber","doi":"10.1109/QRS51102.2020.00019","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00019","url":null,"abstract":"Evaluations of intrusion detection systems (IDS) require log data sets collected in realistic system environments. Ex-isting testbeds therefore offer user simulations and attack scenarios that target specific use-cases. However, not only does the preparation of such testbeds require domain knowledge and time-consuming work, but also maintenance and modifications for other use-cases involve high manual efforts and repeated execution of tasks. We therefore propose to generate testbeds for IDS evaluation using strategies from model-driven engineering. In particular, our approach models system infrastructure, simulated normal behavior, and attack scenarios as testbed-independent modules. A transformation engine then automatically generates arbitrary numbers of testbeds, each with a particular set of characteristics and capable of running in parallel. Our approach greatly improves configurability and flexibility of testbeds and allows to reuse components across multiple scenarios. We use our proof-of-concept implementation to generate a labeled data set for IDS evaluation that is published with this paper.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130869956","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jian K. Liu, Kun Xiao, Lei Luo, Yun Li, Lirong Chen
{"title":"An intrusion detection system integrating network-level intrusion detection and host-level intrusion detection","authors":"Jian K. Liu, Kun Xiao, Lei Luo, Yun Li, Lirong Chen","doi":"10.1109/QRS51102.2020.00028","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00028","url":null,"abstract":"With the rapid development of Internet, the issue of cyber security has increasingly gained more attention. An intrusion Detection System (IDS) is an effective technique to defend cyber-attacks and reduce security losses. However, the challenge of IDS lies in the diversity of cyber-attackers and the frequently-changing data requiring a flexible and efficient solution. To address this problem, machine learning approaches are being applied in the IDS field. In this paper, we propose an efficient scalable neural-network-based hybrid IDS framework with the combination of Host-level IDS (HIDS) and Network-level IDS (NIDS). We applied the autoencoders (AE) to NIDS and designed HIDS using word embedding and convolutional neural network. To evaluate the IDS, many experiments are performed on the public datasets NSL-KDD and ADFA. It can detect many attacks and reduce the security risk with high efficiency and excellent scalability.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"51 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115659031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"PHM Technology for Memory Anomalies in Cloud Computing for IaaS","authors":"Xiwei Qiu, Yuan-Shun Dai, Peng Sun, Xin Jin","doi":"10.1109/QRS51102.2020.00018","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00018","url":null,"abstract":"The IaaS (Infrastructure as a Service) is one of the most popular services from todays cloud service providers, where the virtual machines (VM) are rented by users who can deploy any program they want in the VMs to make their own websites or use as their remote desktops. However, this poses a major challenge for cloud IaaS providers who cannot control the software programs that users develop, install or download on their rented VMs. Those programs may not be well developed with various bugs or even downloaded/installed together with virus, which often make damages to the VMs or infect the cloud platform. To keep the health of a cloud IaaS platform, it is very important to implement the PHM (Prognostics and Health Management) technology for detecting those software problems and self-healing them in an intelligent and timely way. This paper realized a novel PHM technology inspired by biological autonomic nervous system to deal with the memory anomalies of those programs running on the cloud IaaS platform. We first present an innovative autonomic computing technology called Bionic Autonomic Nervous System (BANS) to endow the cloud system with distinctive capabilities of perception, detection, reflection, and learning. Then, we propose a BANS-based Prognostics and Health Management (BPHM) technology to enable the cloud system self-dealing with various memory anomalies. AI-based failure prognostics, immediate self-healing, self-learning ability and self-improvement functions are implemented. Experimental results illustrate that the designed BPHM can automatically and intelligently deal with complex memory anomalies in a real cloud system for IaaS, to keep the system much more reliable and healthier.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134558871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Multi-Objective Learning Method for Building Sparse Defect Prediction Models","authors":"Xin Li, Xiaoxing Yang, Jianmin Su, Wushao Wen","doi":"10.1109/QRS51102.2020.00037","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00037","url":null,"abstract":"Software defect prediction constructs a model from the previous version of a software project to predict defects in the current version, which can help software testers to focus on software modules with more defects in the current version. Most existing methods construct defect prediction models through minimizing the defect prediction error measures. Some researchers proposed model construction approaches that directly optimized the ranking performance in order to achieve an accurate order. In some situations, the model complexity is also considered. Therefore, defect prediction can be seen as a multi-objective optimization problem and should be solved by multi-objective approaches. And hence, in this paper, we employ an existing multi-objective evolutionary algorithm and propose a new multi-objective learning method based on it, to construct defect prediction models by simultaneously optimizing more than one goal. Experimental results over 30 sets of cross-version data show the effectiveness of the proposed multi-objective approaches.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126278088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ai Gong, Yi Zhong, W. Zou, Yangyang Shi, Chunrong Fang
{"title":"Incorporating Android Code Smells into Java Static Code Metrics for Security Risk Prediction of Android Applications","authors":"Ai Gong, Yi Zhong, W. Zou, Yangyang Shi, Chunrong Fang","doi":"10.1109/QRS51102.2020.00017","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00017","url":null,"abstract":"With the wide-spread use of Android applications in people’s daily life, it becomes more and more important to timely identify the security problems of these applications. To enrich existing studies in guarding the security and privacy of Android applications, we attempted to predict the security risk levels of Android applications. Specifically, we proposed an approach that incorporated Android code smells into traditional Java code metrics to predict how secure an Android application is. With an evaluation of our technique on 3,680 Android applications, we found that: (1) Android code smells could help improve the performance of security risk prediction of Android applications; (2) By building a Random Forest model based on Android code smells and Java code metrics, we could achieve an Area Under Curve (AUC) of 0.97; (3) Android code smells such as member ignoring method (MIM) and leaking inner class (LIC) have a relatively-large influence on Android security risk prediction, to which developers should pay more attention during their application development.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121480666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Approach to Determine the Optimal k-Value of K-means Clustering in Adaptive Random Testing","authors":"Jinfu Chen, Lingling Zhao, Minmin Zhou, Yisong Liu, Songling Qin","doi":"10.1109/QRS51102.2020.00032","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00032","url":null,"abstract":"Adaptive Random Testing (ART) aims at improving detection effectiveness by evenly distributing test cases over the whole input domain. Many ART algorithms introducing clustering techniques (such as k-means Clustering) have been proposed to achieve an even spread of test cases. Though previous studies have demonstrated that ART with k-means clustering could achieve a good enhancement in testing effectiveness, k-means clustering is limited by the value of k, which will have a great impact on the test effectiveness. To improve the testing effectiveness of these techniques for object-oriented software, in this paper, we propose an approach named Determination Method of Optimal k-value based on the Experimental Process (DMOVk-EP) to determine the optimal k-value of k-means clustering and make the ART algorithms using k-means clustering technique achieve the best fault detection capability. The proposed method consists of two parts, one is a solution model for k based on the experimental process, and the other is an optimal k-value algorithm based on the presented model. We integrate this method with k-means clustering in ART and apply it to a set of open-source programs, with the experimental results showing that our approach obtains much more appropriate k, and also achieves much better testing effectiveness than other related methods.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133218004","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multi-objective Search for Model-based Testing","authors":"Rui Wang, Cyrille Artho, L. Kristensen, V. Stolz","doi":"10.1109/QRS51102.2020.00029","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00029","url":null,"abstract":"This paper presents a search-based approach relying on multi-objective reinforcement learning and optimization for test case generation in model-based software testing. Our approach considers test case generation as an exploration versus exploitation dilemma, and we address this dilemma by implementing a particular strategy of multi-objective multi-armed bandits with multiple rewards. After optimizing our strategy using the jMetal multi-objective optimization framework, the resulting parameter setting is then used by an extended version of the Modbat tool for model-based testing. We experimentally evaluate our search-based approach on a collection of examples, such as the ZooKeeper distributed service and PostgreSQL database system, by comparing it to the use of random search for test case generation. Our results show that test cases generated using our search-based approach can obtain more predictable and better state/transition coverage, find failures earlier, and provide improved path coverage.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131161977","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuqian Pan, Haichun Zhang, Mingyang Gong, Zhenglin Liu
{"title":"Process-variation Effects on 3D TLC Flash Reliability: Characterization and Mitigation Scheme","authors":"Yuqian Pan, Haichun Zhang, Mingyang Gong, Zhenglin Liu","doi":"10.1109/QRS51102.2020.00051","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00051","url":null,"abstract":"In Solid State Drives, flash management techniques such as wear-leveling and refresh usually assume NAND flash memories have the same endurance value. However, the actual endurance values differ from blocks to blocks. This reliability difference is introduced by process-variation during flash fabrication. In recent years, for improving flash management techniques, various works have been done on the reliability variation of 2D flash memory. As 2D NAND transmitted to 3D NAND flash, the vertical structure and multi-layer stacking changed the effect of previously known reliability problems. In this paper, we are first to characterize the process-variation effects on 3D TLC flash reliability. The characterization includes two parts: endurance variation and error feature variation. Second, we propose an adaptive error prediction scheme to mitigate the process-variation effects. This scheme uses the machine-learning model to realize the error prediction operation. We also discuss the implications of this scheme on main flash management techniques.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115312492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lian Yu, Lijun Liu, Yanbing Jiang, Qi Jing, Bei Zhao, Chen Zhang
{"title":"Attack Graph Auto-Generation for Blockchains based on Bigraphical Reaction Systems","authors":"Lian Yu, Lijun Liu, Yanbing Jiang, Qi Jing, Bei Zhao, Chen Zhang","doi":"10.1109/QRS51102.2020.00046","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00046","url":null,"abstract":"Blockchains (BCs) are claimed to have immutability, distributed consensus, established trust, distributed identity and eternal verifiable, and sound like the ultimate security unim-peachable technology. At the time, however, new age security attacks on the key components of BCs are emerging, which are very sophisticated and can cause huge irreparable damages, including network-based attacks, consensus & ledger-based at-tacks, smart contract-based attacks, and wallet-based attacks. This paper proposes to use bigraph theory to model BC attack meta-model, and automatically generate attack graphs for BC security evaluation. Bigraphical sorting mechanism is used to depict configuration of BC systems, and bigraphical reaction rules are designed to characterize attack templates and attacker behaviours. Adaptive exploit flow approach is proposed to reduce the complexity of matching algorithm guided by interested attack exploits, and probability is introduced into bigraphs to measure the capability of attackers. Preliminary experiments have shown the validity of the proposed approach.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116823585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}