2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)最新文献

筛选

英文中文

Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study 深度神经网络离线和在线测试的比较:一个自动驾驶汽车的案例研究

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-11-28 DOI: 10.1109/ICST46399.2020.00019

Fitash Ul Haq, Donghwan Shin, S. Nejati, L. Briand

{"title":"Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study","authors":"Fitash Ul Haq, Donghwan Shin, S. Nejati, L. Briand","doi":"10.1109/ICST46399.2020.00019","DOIUrl":"https://doi.org/10.1109/ICST46399.2020.00019","url":null,"abstract":"There is a growing body of research on developing testing techniques for Deep Neural Networks (DNNs). We distinguish two general modes of testing for DNNs: Offline testing where DNNs are tested as individual units based on test datasets obtained independently from the DNNs under test, and online testing where DNNs are embedded into a specific application and tested in a close-loop mode in interaction with the application environment. In addition, we identify two sources for generating test datasets for DNNs: Datasets obtained from real-life and datasets generated by simulators. While offline testing can be used with datasets obtained from either sources, online testing is largely confined to using simulators since online testing within real-life applications can be time consuming, expensive and dangerous. In this paper, we study the following two important questions aiming to compare test datasets and testing modes for DNNs: First, can we use simulator-generated data as a reliable substitute to real-world data for the purpose of DNN testing? Second, how do online and offline testing results differ and complement each other? Though these questions are generally relevant to all autonomous systems, we study them in the context of automated driving systems where, as study subjects, we use DNNs automating end-to-end control of cars’ steering actuators. Our results show that simulator-generated datasets are able to yield DNN prediction errors that are similar to those obtained by testing DNNs with real-life datasets. Further, offline testing is more optimistic than online testing as many safety violations identified by online testing could not be identified by offline testing, while large prediction errors generated by offline testing always led to severe safety violations detectable by online testing.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"85 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122661882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 40

ct-fuzz: Fuzzing for Timing Leaks ct-fuzz:定时泄漏的模糊检测

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST) Pub Date : 2019-04-15 DOI: 10.1109/icst46399.2020.00063

Shaobo He, M. Emmi, Gabriela F. Cretu-Ciocarlie

{"title":"ct-fuzz: Fuzzing for Timing Leaks","authors":"Shaobo He, M. Emmi, Gabriela F. Cretu-Ciocarlie","doi":"10.1109/icst46399.2020.00063","DOIUrl":"https://doi.org/10.1109/icst46399.2020.00063","url":null,"abstract":"Testing-based methodologies like fuzzing are able to analyze complex software which is not amenable to traditional formal approaches like verification, model checking, and abstract interpretation. Despite enormous success a texposing countless security vulnerabilities in many popular software projects, applications of testing-based approaches mainly targeted checking traditional safety properties like memory safety. While unquestionably important, this class of properties does not precisely characterize other important security aspects such as information leakage, e.g., through side channels.In this work we extend testing-based software analysis methodologies to two-safety properties, which enables the precise discovery of information leaks in complex software. In particular, we present the ct-fuzz tool, which lends coverage-guided grey box fuzzers the ability to detect two safety property violations. Our approach is capable of exposing violations to any two-safety property expressed a sequality between two program traces. Empirically, we demonstrate that ct-fuzz swiftly reveals timing leaks in popular cryptographic implementations.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"13 6","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113975954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 23

首页上一页