Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis最新文献_第2页

Compiler fuzzing through deep learning 通过深度学习实现编译器模糊测试

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213848

Chris Cummins, Pavlos Petoumenos, A. Murray, Hugh Leather

{"title":"Compiler fuzzing through deep learning","authors":"Chris Cummins, Pavlos Petoumenos, A. Murray, Hugh Leather","doi":"10.1145/3213846.3213848","DOIUrl":"https://doi.org/10.1145/3213846.3213848","url":null,"abstract":"Random program generation — fuzzing — is an effective technique for discovering bugs in compilers but successful fuzzers require extensive development effort for every language supported by the compiler, and often leave parts of the language space untested. We introduce DeepSmith, a novel machine learning approach to accelerating compiler validation through the inference of generative models for compiler inputs. Our approach infers a learned model of the structure of real world code based on a large corpus of open source code. Then, it uses the model to automatically generate tens of thousands of realistic programs. Finally, we apply established differential testing methodologies on them to expose bugs in compilers. We apply our approach to the OpenCL programming language, automatically exposing bugs with little effort on our side. In 1,000 hours of automated testing of commercial and open source compilers, we discover bugs in all of them, submitting 67 bug reports. Our test cases are on average two orders of magnitude smaller than the state-of-the-art, require 3.03× less time to generate and evaluate, and expose bugs which the state-of-the-art cannot. Our random program generator, comprising only 500 lines of code, took 12 hours to train for OpenCL versus the state-of-the-art taking 9 man months to port from a generator for C and 50,000 lines of code. With 18 lines of code we extended our program generator to a second language, uncovering crashes in Solidity compilers in 12 hours of automated testing.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77655783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 108

Evaluating test-suite reduction in real software evolution 评估真实软件演进中的测试套件缩减

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213875

A. Shi, A. Gyori, Suleman Mahmood, Peiyuan Zhao, D. Marinov

{"title":"Evaluating test-suite reduction in real software evolution","authors":"A. Shi, A. Gyori, Suleman Mahmood, Peiyuan Zhao, D. Marinov","doi":"10.1145/3213846.3213875","DOIUrl":"https://doi.org/10.1145/3213846.3213875","url":null,"abstract":"Test-suite reduction (TSR) speeds up regression testing by removing redundant tests from the test suite, thus running fewer tests in the future builds. To decide whether to use TSR or not, a developer needs some way to predict how well the reduced test suite will detect real faults in the future compared to the original test suite. Prior research evaluated the cost of TSR using only program versions with seeded faults, but such evaluations do not explicitly predict the effectiveness of the reduced test suite in future builds. We perform the first extensive study of TSR using real test failures in (failed) builds that occurred for real code changes. We analyze 1478 failed builds from 32 GitHub projects that run their tests on Travis. Each failed build can have multiple faults, so we propose a family of mappings from test failures to faults. We use these mappings to compute Failed-Build Detection Loss (FBDL), the percentage of failed builds where the reduced test suite misses to detect all the faults detected by the original test suite. We find that FBDL can be up to 52.2%, which is higher than suggested by traditional TSR metrics. Moreover, traditional TSR metrics are not good predictors of FBDL, making it difficult for developers to decide whether to use reduced test suites.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90668240","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 28

Shaping program repair space with existing patches and similar code 利用现有补丁和类似代码塑造程序修复空间

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213871

Jiajun Jiang, Yingfei Xiong, Hongyu Zhang, Qing Gao, Xiangqun Chen

{"title":"Shaping program repair space with existing patches and similar code","authors":"Jiajun Jiang, Yingfei Xiong, Hongyu Zhang, Qing Gao, Xiangqun Chen","doi":"10.1145/3213846.3213871","DOIUrl":"https://doi.org/10.1145/3213846.3213871","url":null,"abstract":"Automated program repair (APR) has great potential to reduce bug-fixing effort and many approaches have been proposed in recent years. APRs are often treated as a search problem where the search space consists of all the possible patches and the goal is to identify the correct patch in the space. Many techniques take a data-driven approach and analyze data sources such as existing patches and similar source code to help identify the correct patch. However, while existing patches and similar code provide complementary information, existing techniques analyze only a single source and cannot be easily extended to analyze both. In this paper, we propose a novel automatic program repair approach that utilizes both existing patches and similar code. Our approach mines an abstract search space from existing patches and obtains a concrete search space by differencing with similar code snippets. Then we search within the intersection of the two search spaces. We have implemented our approach as a tool called SimFix, and evaluated it on the Defects4J benchmark. Our tool successfully fixed 34 bugs. To our best knowledge, this is the largest number of bugs fixed by a single technology on the Defects4J benchmark. Furthermore, as far as we know, 13 bugs fixed by our approach have never been fixed by the current approaches.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86195716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 242

Mutode: generic JavaScript and Node.js mutation testing tool 通用的JavaScript和Node.js变异测试工具

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3229504

Diego Rodríguez-Baquero, M. Vásquez

{"title":"Mutode: generic JavaScript and Node.js mutation testing tool","authors":"Diego Rodríguez-Baquero, M. Vásquez","doi":"10.1145/3213846.3229504","DOIUrl":"https://doi.org/10.1145/3213846.3229504","url":null,"abstract":"Mutation testing is a technique in which faults (mutants) are injected into a program or application to assess its test suite effectiveness. It works by inserting mutants and running the application’s test suite to identify if the mutants are detected (killed) or not (survived) by the tests. Although computationally expensive, it has proven to be an effective method to assess application test suites. Several mutation testing frameworks and tools have been built for the various programing languages, however, very few tools have been built for the JavaScript language, more specifically, there is a lack of mutation testing tools for the Node.js runtime and npm based applications. The npm Registry is a public collection of modules of open-source code for Node.js, front-end web applications, mobile applications, robots, routers, and countless other needs of the JavaScript community. The over 700,000 packages hosted in npm are downloaded more than 5 billion times per week. More and more software is published in npm every day, representing a huge opportunity to share code and solutions, but also to share bugs and faulty software. In this paper, we briefly describe prior work for mutation operators in JavaScript and Node.js, and propose Mutode, an open source tool which leverages the npm package ecosystem to perform mutation testing for JavaScript and Node.js applications. We empirically evaluated Mutode effectiveness by running it on 12 of the top 20 npm modules that have automated test suites.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82396525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

Deep specification mining 深规范开采

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213876

Tien-Duy B. Le, D. Lo

{"title":"Deep specification mining","authors":"Tien-Duy B. Le, D. Lo","doi":"10.1145/3213846.3213876","DOIUrl":"https://doi.org/10.1145/3213846.3213876","url":null,"abstract":"Formal specifcations are essential but usually unavailable in software systems. Furthermore, writing these specifcations is costly and requires skills from developers. Recently, many automated techniques have been proposed to mine specifcations in various formats including fnite-state automaton (FSA). However, more works in specifcation mining are needed to further improve the accuracy of the inferred specifcations. In this work, we propose Deep Specifcation Miner (DSM), a new approach that performs deep learning for mining FSA-based specifcations. Our proposed approach uses test case generation to generate a richer set of execution traces for training a Recurrent Neural Network Based Language Model (RNNLM). From these execution traces, we construct a Prefx Tree Acceptor (PTA) and use the learned RNNLM to extract many features. These features are subsequently utilized by clustering algorithms to merge similar automata states in the PTA for constructing a number of FSAs. Then, our approach performs a model selection heuristic to estimate F-measure of FSAs and returns the one with the highest estimated Fmeasure. We execute DSM to mine specifcations of 11 target library classes. Our empirical analysis shows that DSM achieves an average F-measure of 71.97%, outperforming the best performing baseline by 28.22%. We also demonstrate the value of DSM in sandboxing Android apps.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77928592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 44

Translating code comments to procedure specifications 将代码注释翻译成过程规范

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213872

Arianna Blasi, A. Goffi, Konstantin Kuznetsov, Alessandra Gorla, Michael D. Ernst, M. Pezzè, S. D. Castellanos

引用次数: 80

Identifying implementation bugs in machine learning based image classifiers using metamorphic testing 使用变形测试识别基于机器学习的图像分类器中的实现错误

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213858

Anurag Dwarakanath, Manish Ahuja, Samarth Sikand, Raghotham M. Rao, Jagadeesh Chandra J. C. Bose, Neville Dubash, Sanjay Podder

引用次数: 144

Repositioning of static analysis alarms 静态分析告警重新定位

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3213850

Tukaram Muske, Rohith Talluri, Alexander Serebrenik

{"title":"Repositioning of static analysis alarms","authors":"Tukaram Muske, Rohith Talluri, Alexander Serebrenik","doi":"10.1145/3213846.3213850","DOIUrl":"https://doi.org/10.1145/3213846.3213850","url":null,"abstract":"The large number of alarms reported by static analysis tools is often recognized as one of the major obstacles to industrial adoption of such tools. We present repositioning of alarms, a novel automatic postprocessing technique intended to reduce the number of reported alarms without affecting the errors uncovered by them. The reduction in the number of alarms is achieved by moving groups of related alarms along the control flow to a program point where they can be replaced by a single alarm. In the repositioning technique, as the locations of repositioned alarms are different than locations of the errors uncovered by them, we also maintain traceability links between a repositioned alarm and its corresponding original alarm(s). The presented technique is tool-agnostic and orthogonal to many other techniques available for postprocessing alarms. To evaluate the technique, we applied it as a postprocessing step to alarms generated for 4 verification properties on 16 open source and 4 industry applications. The results indicate that the alarms repositioning technique reduces the alarms count by up to 20% over the state-of-the-art alarms grouping techniques with a median reduction of 7.25%.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78722118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 20

MalViz: an interactive visualization tool for tracing malware MalViz:用于跟踪恶意软件的交互式可视化工具

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3229501

V. Nguyen, A. Namin, Tommy Dang

引用次数: 13

Test input generation with Java PathFinder: then and now (invited talk abstract) 使用Java PathFinder生成测试输入:过去和现在(特邀演讲摘要)

Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2018-07-12 DOI: 10.1145/3213846.3234687

S. Khurshid, C. Pasareanu, W. Visser

{"title":"Test input generation with Java PathFinder: then and now (invited talk abstract)","authors":"S. Khurshid, C. Pasareanu, W. Visser","doi":"10.1145/3213846.3234687","DOIUrl":"https://doi.org/10.1145/3213846.3234687","url":null,"abstract":"The paper Test Input Generation With Java PathFinder was published in the International Symposium on Software Testing and Analysis (ISSTA) 2004 Proceedings, and has now been selected to receive the ISSTA 2018 Retrospective Impact Paper Award. The paper described black-box and white-box techniques for the automated testing of software systems. These techniques were based on model checking and symbolic execution and incorporated in the Java PathFinder analysis tool. The main contribution of the paper was to describe how to perform efficient test input generation for code manipulating complex data that takes into account complex method preconditions and evaluate the techniques for generating high coverage tests. We review the original paper and we discuss the research that preceded it and the research that has happened between then (2004) and now (2018) in the context of the Java PathFinder tool, its symbolic execution component that is now called Symbolic PathFinder, and closely related approaches that target testing of software that manipulates complex data structures. We close with directions for future work.","PeriodicalId":20542,"journal":{"name":"Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2018-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74633287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3