Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering最新文献

Audee

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416571

Qianyu Guo, Xiaofei Xie, Yi Li, Xiaoyu Zhang, Yang Liu, Xiaohong Li, Chao Shen

{"title":"Audee","authors":"Qianyu Guo, Xiaofei Xie, Yi Li, Xiaoyu Zhang, Yang Liu, Xiaohong Li, Chao Shen","doi":"10.1145/3324884.3416571","DOIUrl":"https://doi.org/10.1145/3324884.3416571","url":null,"abstract":"Deep learning (DL) has been applied widely, and the quality of DL system becomes crucial, especially for safety-critical applications. Existing work mainly focuses on the quality analysis of DL models, but lacks attention to the underlying frameworks on which all DL models depend. In this work, we propose Audee, a novel approach for testing DL frameworks and localizing bugs. Audee adopts a search-based approach and implements three different mutation strategies to generate diverse test cases by exploring combinations of model structures, parameters, weights and inputs. Audee is able to detect three types of bugs: logical bugs, crashes and Not-a-Number (NaN) errors. In particular, for logical bugs, Audee adopts a cross-reference check to detect behavioural inconsistencies across multiple frameworks (e.g., TensorFlow and PyTorch), which may indicate potential bugs in their implementations. For NaN errors, Audee adopts a heuristic-based approach to generate DNNs that tend to output outliers (i.e., too large or small values), and these values are likely to produce NaN. Furthermore, Audee leverages a causal-testing based technique to localize layers as well as parameters that cause inconsistencies or bugs. To evaluate the effectiveness of our approach, we applied Audee on testing four DL frameworks, i.e., TensorFlow, PyTorch, CNTK, and Theano. We generate a large number of DNNs which cover 25 widely-used APIs in the four frameworks. The results demonstrate that Audee is effective in detecting inconsistencies, crashes and NaN errors. Intotal, 26 unique unknown bugs were discovered, and 7 of them have already been confirmed or fixed by the developers.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122972128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 62

BuildFast BuildFast

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416616

Bihuan Chen, Linlin Chen, Chen Zhang, Xin Peng

{"title":"BuildFast","authors":"Bihuan Chen, Linlin Chen, Chen Zhang, Xin Peng","doi":"10.1145/3324884.3416616","DOIUrl":"https://doi.org/10.1145/3324884.3416616","url":null,"abstract":"Long build times in continuous integration (CI) can greatly increase the cost in human and computing resources, and thus become a common barrier faced by software organizations adopting CI. Build outcome prediction has been proposed as one of the remedies to reduce such cost. However, the state-of-the-art approaches have a poor prediction performance for failed builds, and are not designed for practical usage scenarios. To address the problems, we first conduct an empirical study on 2,590,917 builds to characterize build times in realworld projects, and a survey with 75 developers to understand their perceptions about build outcome prediction. Then, motivated by our study and survey results, we propose a new history-aware approach, named BUILDFAST, to predict CI build outcomes cost-efficiently and practically. We develop multiple failure-specific features from closely related historical builds via analyzing build logs and changed files, and propose an adaptive prediction model to switch between two models based on the build outcome of the previous build. We investigate a practical online usage scenario of BUILDFAST, where builds are predicted in chronological order, and measure the benefit from correct predictions and the cost from incorrect predictions. Our experiments on 20 projects have shown that BUILDFAST improved the state-of-the-art by 47.5% in F1-score for failed builds.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121184150","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

MinerRay MinerRay

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416580

Alan Romano, Yunhui Zheng, Weihang Wang

{"title":"MinerRay","authors":"Alan Romano, Yunhui Zheng, Weihang Wang","doi":"10.1145/3324884.3416580","DOIUrl":"https://doi.org/10.1145/3324884.3416580","url":null,"abstract":"Recent advances in web technology have made in-browser crypto-mining a viable funding model. However, these services have been abused to launch large-scale cryptojacking attacks to secretly mine cryptocurrency in browsers. To detect them, various signature-based or runtime feature-based methods have been proposed. However, they can be imprecise or easily circumvented. To this end, we propose MinerRay, a generic scheme to detect malicious in-browser cryptominers. Instead of leveraging unreliable external patterns, MinerRay infers the essence of cryptomining behaviors that differentiate mining from common browser activities in both WebAssembly and JavaScript contexts. Additionally, to detect stealthy mining activities without user consents, MinerRay checks if the miner can only be instantiated from user actions. MinerRay was evaluated on over 1 million websites. It detected cryptominers on 901 websites, where 885 secretly start mining without user consent. Besides, we compared MinerRay with five state-of-the-art signature-based or behavior-based cryptominer detectors (MineSweeper, CMTracker, Outguard, No Coin, and minerBlock). We observed that emerging miners with new signatures or new services were detected by MinerRay but missed by others. The results show that our proposed technique is effective and robust in detecting evolving cryptominers, yielding more true positives, and fewer errors.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130193440","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 20

OCoR OCoR

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416530

Qihao Zhu, Zeyu Sun, Xiran Liang, Yingfei Xiong, Lu Zhang

{"title":"OCoR","authors":"Qihao Zhu, Zeyu Sun, Xiran Liang, Yingfei Xiong, Lu Zhang","doi":"10.1145/3324884.3416530","DOIUrl":"https://doi.org/10.1145/3324884.3416530","url":null,"abstract":"Code retrieval helps developers reuse code snippets in the open-source projects. Given a natural language description, code retrieval aims to search for the most relevant code relevant among a set of code snippets. Existing state-of-the-art approaches apply neural networks to code retrieval. However, these approaches still fail to capture an important feature: overlaps. The overlaps between different names used by different people indicate that two different names may be potentially related (e.g., “message” and “msg”), and the overlaps between identifiers in code and words in natural language descriptions indicate that the code snippet and the description may potentially be related. To address this problem, we propose a novel neural architecture named OCoR, where we introduce two specifically-designed components to capture overlaps: the first embeds names by characters to capture the overlaps between names, and the second introduces a novel overlap matrix to represent the degrees of overlaps between each natural language word and each identifier. The evaluation was conducted on two established datasets. The experimental results show that OCoR significantly outperforms the existing state-of-the-art approaches and achieves 13.1% to 22.3% improvements. Moreover, we also conducted several in-depth experiments to help understand the performance of the different components in OCoR.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115209147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 26

FILO 锐利

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3415290

M. Mobilio, O. Riganelli, D. Micucci, L. Mariani

引用次数: 3

ChemTest ChemTest

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416638

Michael C. Gerten, James I. Lathrop, Myra B. Cohen, T. Klinge

{"title":"ChemTest","authors":"Michael C. Gerten, James I. Lathrop, Myra B. Cohen, T. Klinge","doi":"10.1145/3324884.3416638","DOIUrl":"https://doi.org/10.1145/3324884.3416638","url":null,"abstract":"In recent years the use of non-traditional computing mechanisms has grown rapidly. One paradigm uses chemical reaction networks (CRNs) to compute via chemical interactions. CRNs are used to prototype molecular devices at the nanoscale such as intelligent drug therapeutics. In practice, these programs are first written and simulated in environments such as MatLab and later compiled into physical molecules such as DNA strands. However, techniques for testing the correctness of CRNs are lacking. Current methods of validating CRNs include model checking and theorem proving, but these are limited in scalability. In this paper we present the first (to the best of our knowledge) testing framework for CRNs, ChemTest. ChemTest evaluates test oracles on individual simulation traces and supports functional, metamorphic, internal and hyper test cases. It also allows for flakiness and programs that are probabilistic. We performed a large case study demonstrating that ChemTest can find seeded faults and scales beyond model checking. Of our tests, 21% are inherently flaky, suggesting that systematic support for this paradigm is needed. On average, functional tests find 66.5% of the faults, while metamorphic tests find 80.4%, showing the benefit of using metamorphic relationships in our test framework. In addition, we show how the time at evaluation impacts fault detection.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115730452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

MoFuzz

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416668

Hoang Lam Nguyen, Nebras Nassar, Timo Kehrer, Lars Grunske

{"title":"MoFuzz","authors":"Hoang Lam Nguyen, Nebras Nassar, Timo Kehrer, Lars Grunske","doi":"10.1145/3324884.3416668","DOIUrl":"https://doi.org/10.1145/3324884.3416668","url":null,"abstract":"Fuzzing or fuzz testing is an established technique that aims to discover unexpected program behavior (e.g., bugs, security vulnerabilities, or crashes) by feeding automatically generated data into a program under test. However, the application of fuzzing to test Model-Driven Software Engineering (MDSE) tools is still limited because of the difficulty of existing fuzzers to provide structured, well-typed inputs, namely models that conform to typing and consistency constraints induced by a given meta-model and underlying modeling framework. By drawing from recent advances on both fuzz testing and automated model generation, we present three different approaches for fuzzing MDSE tools: A graph grammar-based fuzzer and two variants of a coverage-guided mutation-based fuzzer working with different sets of model mutation operators. Our evaluation on a set of real-world MDSE tools shows that our approaches can outperform both standard fuzzers and model generators w.r.t. their fuzzing capabilities. Moreover, we found that each of our approaches comes with its own strengths and weaknesses in terms of fault finding capabilities and the ability to cover different aspects of the system under test. Thus the approaches complement each other, forming a fuzzer suite for testing MDSE tools.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115129328","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

CoFI fishing

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3416548

Haicheng Chen, Wensheng Dou, Dong Wang, Feng Qin

{"title":"CoFI","authors":"Haicheng Chen, Wensheng Dou, Dong Wang, Feng Qin","doi":"10.1145/3324884.3416548","DOIUrl":"https://doi.org/10.1145/3324884.3416548","url":null,"abstract":"Network partitions are inevitable in large-scale cloud systems. Despite developer's efforts in handling network partitions throughout designing, implementing and testing cloud systems, bugs caused by network partitions, i.e., partition bugs, still exist and cause severe failures in production clusters. It is challenging to expose these partition bugs because they often require network partitions to start and stop at specific timings. In this paper, we propose Consistency-Guided Fault Injection (CoFI), a novel technique that systematically injects network partitions to effectively expose partition bugs. We observe that, network partitions can leave cloud systems in inconsistent states, where partition bugs are more likely to occur. Based on this observation, CoFI first infers invariants (i.e., consistent states) among different nodes in a cloud system. Once detecting violations to the inferred invariants (i.e., inconsistent states) while running the cloud system, CoFI injects network partitions to prevent the cloud system from recovering back to consistent states, and thoroughly tests whether the cloud system still proceeds correctly at inconsistent states. We have applied CoFI to three widely-deployed cloud systems, i.e., Cassandra, HDFS, and YARN. CoFI has detected 12 previously-unknown bugs, and four of them have been confirmed by developers.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125991540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

HomoTR

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3415296

Chenqian Zhu, Weisong Sun, Qin Liu, Yangyang Yuan, Chunrong Fang, Yong Huang

{"title":"HomoTR","authors":"Chenqian Zhu, Weisong Sun, Qin Liu, Yangyang Yuan, Chunrong Fang, Yong Huang","doi":"10.1145/3324884.3415296","DOIUrl":"https://doi.org/10.1145/3324884.3415296","url":null,"abstract":"A growing number of new technologies are used in test development. Among them, automatic test generation, a promising technology to improve the efficiency of unit testing, currently performs not satisfactory in practice. Test recommendation, like code recommendation, is another feasible technology for supporting efficient unit testing and gets increasing attention. In this paper, we develop a novel system, namely HomoTR, which implements online test recommendations by measuring the homology of two methods. If the new method under test shares homology with an existing method that has test cases, HomoTR will recommend the test cases to the new method. The preliminary experiments show that HomoTR can quickly and effectively recommend test cases to help the developers improve the testing efficiency. Besides, HomoTR has been integrated into the MoocTest platform successfully, so it can also execute the recommended test cases automatically and visualize the testing results (e.g., Branch Coverage) friendly to help developers understand the process of testing. The demo video of HomoTR can be found at https://youtu.be/_227EfcUbus.","PeriodicalId":267160,"journal":{"name":"Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122349451","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

AirMochi AirMochi

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Pub Date : 2020-12-21 DOI: 10.1145/3324884.3415304

Nikola Lukic, Saghar Talebipour, N. Medvidović

引用次数: 2