M. Gavrilov, Kyle Dewey, Alex Groce, Davina J. Zamanzadeh, B. Hardekopf
{"title":"A Practical, Principled Measure of Fuzzer Appeal: A Preliminary Study","authors":"M. Gavrilov, Kyle Dewey, Alex Groce, Davina J. Zamanzadeh, B. Hardekopf","doi":"10.1109/QRS51102.2020.00071","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00071","url":null,"abstract":"Fuzzers are important bug-finding tools in both academia and industry. To ensure scientific progress, we need a metric for fuzzer comparison. Bug-based metrics are impractical because (1) the definition of \"bug\" is vague, and (2) mapping bug-revealing inputs to bugs requires extensive domain knowledge.In this paper, we propose an automated method for comparing fuzzers that alleviates these problems. We replace the question \"What bugs can this fuzzer find?\" with \"What changes in program behavior over time can this fuzzer detect?\". Intuitively, fuzzers which find more behavioral changes are likely to find more bugs. However, unlike bugs, behavioral changes are well-defined and readily detectable. Our evaluation, executed on three targets with several fuzzers, shows that our method is consistent with bug-based metrics, but without associated difficulties. While further evaluation is needed to establish superiority, our results show that our method warrants further investigation.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121041201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
P. Dovgalyuk, I. Vasiliev, N. Fursova, D. Dmitriev, Mikhail Abakumov, V. Makarov
{"title":"Non-intrusive Virtual Machine Analysis and Reverse Debugging with SWAT","authors":"P. Dovgalyuk, I. Vasiliev, N. Fursova, D. Dmitriev, Mikhail Abakumov, V. Makarov","doi":"10.1109/QRS51102.2020.00036","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00036","url":null,"abstract":"This paper presents SWAT — System-Wide Analysis Toolkit. It is based on open source emulation and debugging projects and implements the approaches for non-intrusive system-wide analysis and debugging: lightweight OS-agnostic virtual machine introspection, full system execution replay, non-intrusive debugging with WinDbg, and full system reverse debugging. These features are based on novel non-intrusive introspection and reverse debugging methods. They are useful for stealth debugging and analysis of the platforms with custom kernels. SWAT includes multi-platform emulator QEMU with additional instrumentation and debugging features, GUI for convenient QEMU setup and execution, QEMU plugin for non-intrusive introspection, and modified version of GDB. Our toolkit may be useful for the developers of the virtual platforms, emulators, and firmwares/drivers/operating systems. Virtual machine intospection approach does not require loading any guest agents and source code of the OS. Therefore it may be applied to ROM-based guest systems and enables using of record/replay of the system execution. This paper includes the description of SWAT components, analysis methods, and some SWAT use cases.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123989772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Runfeng Mao, He Zhang, Qiming Dai, Huang Huang, Guoping Rong, Haifeng Shen, Lianping Chen, Kaixiang Lu
{"title":"Preliminary Findings about DevSecOps from Grey Literature","authors":"Runfeng Mao, He Zhang, Qiming Dai, Huang Huang, Guoping Rong, Haifeng Shen, Lianping Chen, Kaixiang Lu","doi":"10.1109/QRS51102.2020.00064","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00064","url":null,"abstract":"Context: Emerging from the agile culture, DevOps particularly emphasizes development and deployment speed to achieve rapid value delivery, which however brings some security risks to the software development process. DevSecOps is an extension of DevOps, which is considered as a means to intertwine development, operation and security. Some companies with security concerns begin to take DevSecOps into consideration when it comes to the application of DevOps. Objective: The goal of this study is to report the state-of-the-practice of DevSecOps as well as calling for academia to pay more attention to DevSecOps. Method: Using Google search engine to collect articles on DevSecOps, we conducted a Grey Literature Review (GLR) on the selected articles. Results: Whilst there exists three major software security risks in DevOps, the establishment of DevOps pipeline provides opportunities for software security activities. Based on the preliminary consensus that DevSecOps is an extension of DevOps, it is observed that the interpretations of DevSecOps can be classified into three core aspects, which are: DevSecOps capabilities, cultural enablers, and technological enablers. Furthermore, to materialize the interpretations into daily software production activities, the recommended DevSecOps practices we obtain from Grey Literature (GL) can be categorized in terms of process, infrastructure and collaboration. Conclusion: Although DevSecOps is getting increasing attention by industry, it is still in its infancy and needs to be promoted by both academia and industry.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117313002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Similarity Integration Method based Information Retrieval and Word Embedding in Bug Localization","authors":"Shasha Cheng, Xuefeng Yan, A. Khan","doi":"10.1109/QRS51102.2020.00034","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00034","url":null,"abstract":"To improve the performance of bug localization, there is necessity to solve the lexical mismatch between the natural language in the bug report and the programming language in the source file. A similarity integration method for bug localization is proposed, in which the similarity between bug report and source file is calculated by information retrieval (IR) and word embedding. More specifically, IR technique is used to collect the exact matches between bug report and source file. The terms in the bug report and the potential source files of different code tokens are connected by word embedding technique, which is used to complement with IR technique. Finally, deep neural network (DNN) is utilized to integrate extracted features to get the correlation between bug reports and source files. The experimental results show that the proposed approach outperforms several existing bug localization approaches in terms of Top N Rank, MAP, and MRR.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"105 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131408550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the Impact of Inter-language Dependencies in Multi-language Systems","authors":"Manel Grichi, Mouna Abidi, Fehmi Jaafar, Ellis E. Eghan, Bram Adams","doi":"10.1109/QRS51102.2020.00070","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00070","url":null,"abstract":"Nowadays, developers are often using multiple programming languages to exploit the advantages of each language and to reuse code. However, dependency analysis across multi-language is more challenging compared to mono-language systems. In this paper, we introduce two approaches for multi- language dependency analysis: S-MLDA (Static Multi-language Dependency Analyzer) and H-MLDA (Historical Multi-language Dependency Analyzer), which we apply on ten open-source multi-language systems to empirically analyze the prevalence of the dependencies across languages i.e., inter-language dependencies and their impact on software quality and security.Our main results show that: the more inter-language dependencies, the higher the risk of bugs and vulnerabilities being introduced, while this risk remains constant for intra-language dependencies; the percentage of bugs within inter-language dependencies is three times higher than the percentage of bugs identified in intra-language dependencies; the percentage of vulnerabilities within inter-language dependencies is twice the percentage of vulnerabilities introduced in intra-language dependencies","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"46 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114307851","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jincheng He, Sitao Min, Kelechi Ogudu, Michael Shoga, A. Polak, Iordanis Fostiropoulos, B. Boehm, Pooyan Behnamghader
{"title":"The Characteristics and Impact of Uncompilable Code Changes on Software Quality Evolution","authors":"Jincheng He, Sitao Min, Kelechi Ogudu, Michael Shoga, A. Polak, Iordanis Fostiropoulos, B. Boehm, Pooyan Behnamghader","doi":"10.1109/QRS51102.2020.00061","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00061","url":null,"abstract":"Software repositories allow multiple developers to iteratively contribute commits, with the intention of improving the system. However, commits can negatively impact software quality, or even cause the software to become uncompilable. Recent studies show that uncompilable commits exist even in high-profile open-source software. Identifying broken code, a potential symptom of careless development, and analyzing how software changes when it becomes uncompilable can shed light on how software quality evolves when developers do not follow best practices. Since comprehensive software quality analysis tools are incapable of analyzing uncompilable commits, there is little insight as to what happens and how quality changes when a commit breaks the compilability. In this paper, starting from an analysis of the software quality metric changes that happen when the project become uncompilable, we explore the purposes of commits and the relations between commit type, size and compilability, analyzed across 68 open-source Java repositories.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125434108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Lightweight Fault Localization Approach based on XGBoost","authors":"Bo Yang, Yuze He, Huai Liu, Yixin Chen, Zhi Jin","doi":"10.1109/QRS51102.2020.00033","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00033","url":null,"abstract":"Software fault localization is one of the key activities in software debugging. The program spectrum-based approach is widely used in fault localization. However, lots of program information, for example, the sequence of the execution statement and statement semantics, is missing when such an approach is utilized, which affects the performance. XGBoost is an effective learning algorithm, which can use the characteristics of the training data to build a classification tree during training. In addition, XGBoost can iteratively adjust the information value of the feature, so that the training process retains the importance information of the feature. This paper proposes applying XGBoost into fault localization utilizing information of program execution behaviors. A novel method called XGB-FL is developed, where the program spectrum information is converted into a coverage matrix to train the XGBoost model. We can get the characteristics of the data through the trained model and the importance of the program statement in the classification process. This is also the basis for judging whether the statement is likely to contain a fault. Nine representative data sets have been chosen to evaluate the performance of XGB-FL. The experimental results show that XGB-FL can generally deliver a higher performance in fault localization than those baseline techniques, in terms of precision and efficiency.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121461182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yi Yao, Yuchan Liu, Song Huang, Hao Chen, Jialuo Liu, Fan Yang
{"title":"Cross-Project Dynamic Defect Prediction Model for Crowdsourced test","authors":"Yi Yao, Yuchan Liu, Song Huang, Hao Chen, Jialuo Liu, Fan Yang","doi":"10.1109/QRS51102.2020.00040","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00040","url":null,"abstract":"By comparing the predicted number of defects with the number found in crowdsourced test in real time, people can dynamically assess the progress of crowdsourced test tasks. In this paper, we propose a cross-project dynamic defect prediction model (CPDDPM) for crowdsourced test to predict the number of defects in real time. In the construction of training dataset, we use density-based clustering method to select instances from the multiple source project datasets and build the initial training dataset. In the dynamic correction, CPDDPM iteratively corrects the prediction model using crowdsourced test reports and ability attributes of the crowdsourced testers until the predicted results converge. We collected project defect datasets on the crowdsourced test platform, and evaluated prediction accuracy of CPDDPM by using relative error and prediction at level l. The results show that CPDDPM can greatly improve the prediction performance of defect number.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126220754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Empirical Study of Utilization of Imperative Modules in Ansible","authors":"Shoma Kokuryo, Masanari Kondo, O. Mizuno","doi":"10.1109/QRS51102.2020.00063","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00063","url":null,"abstract":"In recent years, a configuration management tool is adopted to manage complicated and huge systems such as bare-metal servers, cloud computing resources and our personal computers. Such a tool makes the operations to deploy services more efficient and eliminates dependencies on the specific system operators. The operations are required to be idempotent for reproducible deployment. However, the imperative modules whose operations may not be idempotent are used frequently to execute user-defined scripts on the target system; it is unclear why and how they are used, though using them frequently is believed to be a bad practice.In this paper, we studied why and how imperative modules are used in a configuration management tool, Ansible. We found that imperative modules are mainly used to perform operations that are not supported by Ansible, and about 45% of imperative modules are replaceable by other modules; the replaceable modules might be idempotent. We, therefore, recommend developers to look at replaceable modules before using imperative modules since replaceable modules might make their operations idempotent.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126577009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A log-based anomaly detection method with the NW ensemble rules","authors":"Bingming Wang, Shi Ying, Guoli Cheng, Yiyao Li","doi":"10.1109/QRS51102.2020.00022","DOIUrl":"https://doi.org/10.1109/QRS51102.2020.00022","url":null,"abstract":"Log analysis can be used for software system anomaly detection, and ensemble learning can handle log data with imbalanced characteristics. Therefore, log-based anomaly detection with ensemble learning is a good choice. However, the existing data balancing methods used in ensemble learning may destroy the distribution of the original log data and affect the accuracy of the anomaly detection results. Besides, the existing ensemble rules do not take into account the relationship between the samples to be detected and the historical log data. Therefore, we propose a log-based anomaly detection method with the NW (Neighbor Weighting) ensemble rules, which uses a data balancing method based on spectral clustering so that the balanced log data can maintain the distribution of the original data and meet the quantity balance at the same time. Then, a new group of ensemble rules is proposed and used for anomaly detection with higher accuracy. We performed experiments on six large log data sets with different types of systems and verified the feasibility and universality of the method in this paper.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126667801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}