Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis最新文献_第4页

Learning user interface element interactions 学习用户界面元素交互

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3330569

Christian Degott, N. P. Borges, A. Zeller

引用次数: 39

DeepHunter: a coverage-guided fuzz testing framework for deep neural networks 深度神经网络的覆盖引导模糊测试框架

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3330579

Xiaofei Xie, L. Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, S. See

{"title":"DeepHunter: a coverage-guided fuzz testing framework for deep neural networks","authors":"Xiaofei Xie, L. Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, S. See","doi":"10.1145/3293882.3330579","DOIUrl":"https://doi.org/10.1145/3293882.3330579","url":null,"abstract":"The past decade has seen the great potential of applying deep neural network (DNN) based software to safety-critical scenarios, such as autonomous driving. Similar to traditional software, DNNs could exhibit incorrect behaviors, caused by hidden defects, leading to severe accidents and losses. In this paper, we propose DeepHunter, a coverage-guided fuzz testing framework for detecting potential defects of general-purpose DNNs. To this end, we first propose a metamorphic mutation strategy to generate new semantically preserved tests, and leverage multiple extensible coverage criteria as feedback to guide the test generation. We further propose a seed selection strategy that combines both diversity-based and recency-based seed selection. We implement and incorporate 5 existing testing criteria and 4 seed selection strategies in DeepHunter. Large-scale experiments demonstrate that (1) our metamorphic mutation strategy is useful to generate new valid tests with the same semantics as the original seed, by up to a 98% validity ratio; (2) the diversity-based seed selection generally weighs more than recency-based seed selection in boosting the coverage and in detecting defects; (3) DeepHunter outperforms the state of the arts by coverage as well as the quantity and diversity of defects identified; (4) guided by corner-region based criteria, DeepHunter is useful to capture defects during the DNN quantization for platform migration.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75516775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 306

Identifying error code misuses in complex system 复杂系统误码识别

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3338986

Wensheng Tang

{"title":"Identifying error code misuses in complex system","authors":"Wensheng Tang","doi":"10.1145/3293882.3338986","DOIUrl":"https://doi.org/10.1145/3293882.3338986","url":null,"abstract":"Many complex software systems use error codes to differentiate error states. Therefore, it is crucial to ensure those error codes are used correctly. Misuses of error codes can lead to hardly sensible but fatal system failures. These errors are especially difficult to debug, since the failure points are usually far away from the root causes. Existing static analysis approaches to detecting error handling bugs mainly focus on how an error code is propagated or used in a program. However, they do not consider whether an error code is correctly chosen for propagation or usage within different program contexts, and thus miss to detect many error code misuse bugs. In this work, we conduct an empirical study on error code misuses in a mature commercial system. We collect error code issues from the commit history and conclude three main causes of them. To further resolve this problem, we propose a static approach that can automatically detect error code misuses. Our approach takes error code definition and error domain assignment as the input, and uses a novel static analysis method to detect the occurrence of the three categories of error code misuses in the source code.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"115 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75651376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Automatically testing self-driving cars with search-based procedural content generation 通过基于搜索的程序内容生成自动测试自动驾驶汽车

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3330566

Alessio Gambi, Marc Müller, G. Fraser

{"title":"Automatically testing self-driving cars with search-based procedural content generation","authors":"Alessio Gambi, Marc Müller, G. Fraser","doi":"10.1145/3293882.3330566","DOIUrl":"https://doi.org/10.1145/3293882.3330566","url":null,"abstract":"Self-driving cars rely on software which needs to be thoroughly tested. Testing self-driving car software in real traffic is not only expensive but also dangerous, and has already caused fatalities. Virtual tests, in which self-driving car software is tested in computer simulations, offer a more efficient and safer alternative compared to naturalistic field operational tests. However, creating suitable test scenarios is laborious and difficult. In this paper we combine procedural content generation, a technique commonly employed in modern video games, and search-based testing, a testing technique proven to be effective in many domains, in order to automatically create challenging virtual scenarios for testing self-driving car soft- ware. Our AsFault prototype implements this approach to generate virtual roads for testing lane keeping, one of the defining features of autonomous driving. Evaluation on two different self-driving car software systems demonstrates that AsFault can generate effective virtual road networks that succeed in revealing software failures, which manifest as cars departing their lane. Compared to random testing AsFault was not only more efficient, but also caused up to twice as many lane departures.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78311738","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 151

JNI program analysis with automatically extracted C semantic summary JNI程序分析用C语言自动提取语义摘要

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3338990

Sungho Lee

{"title":"JNI program analysis with automatically extracted C semantic summary","authors":"Sungho Lee","doi":"10.1145/3293882.3338990","DOIUrl":"https://doi.org/10.1145/3293882.3338990","url":null,"abstract":"From Oracle JVM to Android Runtime, most Java runtime environments officially support Java Native Interface (JNI) for interaction between Java and C. Using JNI, developers can improve Java program performance or reuse existing libraries implemented in C. At the same time, differences between the languages can lead to various kinds of unexpected bugs when developers do not understand the differences or comprehensive interoperation semantics completely. Furthermore, existing program analysis techniques do not cover the interoperation, which can reduce the quality of JNI programs. We propose a JNI program analysis technique that analyzes Java and C code of JNI programs using analyzers targeting each language respectively. The C analyzer generates a semantic summary for each C function callable from Java and the Java analyzer constructs call graphs using the semantic summaries and Java code. In addition to the call graph construction, we extend the analysis technique to detect four bug types that can occur in the interoperation between the languages. We believe that our approach would be able to detect genuine bugs as well as improve the quality of JNI programs.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"77 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83863673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Some challenges for software testing research (invited talk paper) 软件测试研究面临的一些挑战(特邀报告)

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3338991

N. Alshahwan, Andrea Ciancone, M. Harman, Yue Jia, Ke Mao, Alexandru Marginean, A. Mols, Hila Peleg, Federica Sarro, Ilya Zorin

引用次数: 8

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis 第28届ACM SIGSOFT软件测试与分析国际研讨会论文集

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882

引用次数: 1

Ukwikora: continuous inspection for keyword-driven testing Ukwikora:关键字驱动测试的连续检查

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3339003

Renaud Rwemalika, Marinos Kintis, Mike Papadakis, Yves Le Traon, Pierre Lorrach

{"title":"Ukwikora: continuous inspection for keyword-driven testing","authors":"Renaud Rwemalika, Marinos Kintis, Mike Papadakis, Yves Le Traon, Pierre Lorrach","doi":"10.1145/3293882.3339003","DOIUrl":"https://doi.org/10.1145/3293882.3339003","url":null,"abstract":"Automation of acceptance test suites becomes necessary in the context of agile software development practices, which require rapid feedback on the quality of code changes. To this end, companies try to automate their acceptance tests as much as possible. Unfortunately, the growth of the automated test suites, by several automation testers, gives rise to potential test smells, i.e., poorly designed test code, being introduced in the test code base, which in turn may increase the cost of maintaining the code and creating new one. In this paper, we investigate this problem in the context of our industrial partner, BGL BNP Paribas, and introduce Ukwikora, an automated tool that statically analyzes acceptance test suites, enabling the continuous inspection of the test code base. Ukwikora targets code written in the Robot Framework syntax, a popular framework for writing Keyword-Driven tests. Ukwikora has been successfully deployed at BGL BNP Paribas, detecting issues otherwise unknown to the automation testers, such as the presence of duplicated test code, dead test code and dependency issues among the tests. The success of our case study reinforces the need for additional research and tooling for acceptance test suites.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75370375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Interactive metamorphic testing of debuggers 调试器的交互式变形测试

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3330567

S. Tolksdorf, Daniel Lehmann, Michael Pradel

{"title":"Interactive metamorphic testing of debuggers","authors":"S. Tolksdorf, Daniel Lehmann, Michael Pradel","doi":"10.1145/3293882.3330567","DOIUrl":"https://doi.org/10.1145/3293882.3330567","url":null,"abstract":"When improving their code, developers often turn to interactive debuggers. The correctness of these tools is crucial, because bugs in the debugger itself may mislead a developer, e.g., to believe that executed code is never reached or that a variable has another value than in the actual execution. Yet, debuggers are difficult to test because their input consists of both source code and a sequence of debugging actions, such as setting breakpoints or stepping through code. This paper presents the first metamorphic testing approach for debuggers. The key idea is to transform both the debugged code and the debugging actions in such a way that the behavior of the original and the transformed inputs should differ only in specific ways. For example, adding a breakpoint should not change the control flow of the debugged program. To support the interactive nature of debuggers, we introduce interactive metamorphic testing. It differs from traditional metamorphic testing by determining the input transformation and the expected behavioral change it causes while the program under test is running. Our evaluation applies the approach to the widely used debugger in the Chromium browser, where it finds eight previously unknown bugs with a true positive rate of 51%. All bugs have been confirmed by the developers, and one bug has even been marked as release-blocking.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"44 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82025584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 23

Effective and efficient API misuse detection via exception propagation and search-based testing 通过异常传播和基于搜索的测试进行有效和高效的API误用检测

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis Pub Date : 2019-07-10 DOI: 10.1145/3293882.3330552

M. Kechagia, Xavier Devroey, Annibale Panichella, Georgios Gousios, A. Deursen

{"title":"Effective and efficient API misuse detection via exception propagation and search-based testing","authors":"M. Kechagia, Xavier Devroey, Annibale Panichella, Georgios Gousios, A. Deursen","doi":"10.1145/3293882.3330552","DOIUrl":"https://doi.org/10.1145/3293882.3330552","url":null,"abstract":"Application Programming Interfaces (APIs) typically come with (implicit) usage constraints. The violations of these constraints (API misuses) can lead to software crashes. Even though there are several tools that can detect API misuses, most of them suffer from a very high rate of false positives. We introduce Catcher, a novel API misuse detection approach that combines static exception propagation analysis with automatic search-based test case generation to effectively and efficiently pinpoint crash-prone API misuses in client applications. We validate Catcher against 21 Java applications, targeting misuses of the Java platform's API. Our results indicate that Catcher is able to generate test cases that uncover 243 (unique) API misuses that result in crashes. Our empirical evaluation shows that Catcher can detect a large number of misuses (77 cases) that would remain undetected by the traditional coverage-based test case generator EvoSuite. Additionally, on average, Catcher is eight times faster than EvoSuite in generating test cases for the identified misuses. Finally, we find that the majority of the exceptions triggered by Catcher are unexpected to developers, i.e., not only unhandled in the source code but also not listed in the documentation of the client applications.","PeriodicalId":20624,"journal":{"name":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","volume":"14 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88009723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7