Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings最新文献_第3页

Debugging inputs 调试输入

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings Pub Date : 2020-06-27 DOI: 10.1145/3377812.3390797

Lukas Kirschner, E. Soremekun, A. Zeller

引用次数: 0

EvalDNN

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings Pub Date : 2020-06-27 DOI: 10.1145/3377812.3382133

Yongqiang Tian, Zhihua Zeng, Ming Wen, Yepang Liu, Tzu-yang Kuo, S. Cheung

{"title":"EvalDNN","authors":"Yongqiang Tian, Zhihua Zeng, Ming Wen, Yepang Liu, Tzu-yang Kuo, S. Cheung","doi":"10.1145/3377812.3382133","DOIUrl":"https://doi.org/10.1145/3377812.3382133","url":null,"abstract":"Recent studies have shown that the performance of deep learning models should be evaluated using various important metrics such as robustness and neuron coverage, besides the widely-used prediction accuracy metric. However, major deep learning frameworks currently only provide APIs to evaluate a model’s accuracy. In order to comprehensively assess a deep learning model, framework users and researchers often need to implement new metrics by themselves, which is a tedious job. What is worse, due to the large number of hyper-parameters and inadequate documentation, evaluation results of some deep learning models are hard to reproduce, especially when the models and metrics are both new.To ease the model evaluation in deep learning systems, we have developed EvalDNN, a user-friendly and extensible toolbox supporting multiple frameworks and metrics with a set of carefully designed APIs. Using EvalDNN, evaluation of a pre-trained model with respect to different metrics can be done with a few lines of code. We have evaluated EvalDNN on 79 models from TensorFlow, Keras, GluonCV, and PyTorch. As a result of our effort made to reproduce the evaluation results of existing work, we release a performance benchmark of popular models, which can be a useful reference to facilitate future research. The tool and benchmark are available at https://github.com/yqtianust/EvalDNN and https://yqtianust.github.io/EvalDNN-benchmark/, respectively. A demo video of EvalDNN is available at: https://youtu.be/v69bNJN2bJc.","PeriodicalId":421517,"journal":{"name":"Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126404777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

BigTest

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings Pub Date : 2020-06-27 DOI: 10.1145/3377812.3382145

Muhammad Ali Gulzar, M. Musuvathi, Miryung Kim

{"title":"BigTest","authors":"Muhammad Ali Gulzar, M. Musuvathi, Miryung Kim","doi":"10.1145/3377812.3382145","DOIUrl":"https://doi.org/10.1145/3377812.3382145","url":null,"abstract":"Data-intensive scalable computing (DISC) systems such as Google’s MapReduce, Apache Hadoop, and Apache Spark are prevalent in many production services. Despite their popularity, the quality of DISC applications suffers due to a lack of exhaustive and automated testing. Current practices of testing DISC applications are limited to using a small random sample of the entire input dataset which merely exposes any program faults. Unlike SQL queries, testing DISC applications has new challenges due to a composition of both dataflow and relational operators, and user-defined functions (UDF) that could be arbitrarily long and complex.To address this problem, we demonstrate a new white-box testing framework called BigTest that takes an Apache Spark program as input and automatically generates synthetic, concrete data for effective and efficient testing. BigTest combines the symbolic execution of UDFs with the logical specifications of dataflow and relational operators to explore all paths in a DISC application. Our experiments show that BigTest is capable of generating test data that can reveal up to 2X more faults than the entire data set with 194X less testing time. We implement BigTest in a Java-based command line tool with a pre-compile binary jar. It exposes a configuration file in which a user can edit preferences, including the path of a target program, the upper bound of loop exploration, and a choice of theorem solver. The demonstration video of BigTest is available at https://youtu.be/OeHhoKiDYso and BigTest is available at https://github.com/maligulzar/BigTest.","PeriodicalId":421517,"journal":{"name":"Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120977361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Nimbus

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings Pub Date : 2020-06-27 DOI: 10.1163/2589-7993_eeco_sim_00002398

Robert Chatley, Thomas Allerton

引用次数: 0

PROMISE 承诺

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings Pub Date : 2019-09-03 DOI: 10.2307/j.ctvpb3wdm.6

Sergio García, Patrizio Pelliccione, C. Menghi, T. Berger, T. Bures

引用次数: 0