{"title":"Investigating the Configurations of an Industrial Path Planner in Terms of Collision Avoidance","authors":"Xiaoyi Zhang, Paolo Arcaini, F. Ishikawa, Kun Liu","doi":"10.1109/ISSRE5003.2020.00036","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00036","url":null,"abstract":"Typical approaches to test Autonomous Driving Systems (ADS) generate tests in a simulation environment. A common goal in ADS testing is to find scenarios in which the car collides, as these could witness ADS faults. Recent approaches not only find a collision, but they also show whether it could be avoided: they search for a different ADS configuration (i.e., the setting of some parameters) using which the car does not collide. However, such techniques do not explain why the collision occurs and why the alternative configuration is able to avoid it. In this paper, we propose an approach to investigate the relationship between the ADS configurations and the obtained safety during driving. We first use a technique based on fuzzification to partition ADS parameters in different categories, and a spectra- based analysis to identify which categories relate to hazard and safety. Then, we consider collision scenarios by inspecting how the different ADS configurations affect the driving characteristics (e.g., acceleration and curvature) and, so, cause or avoid a collision. We applied the approach to the path planner of our industry partner, by considering three traffic situations. We observed that the path planner, to guarantee safety, should be configured differently in different situations.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"119 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122674668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Aliénor Damien, Pierre-François Gimenez, N. Feyt, V. Nicomette, M. Kaâniche, E. Alata
{"title":"On-board Diagnosis: A First Step from Detection to Prevention of Intrusions on Avionics Applications","authors":"Aliénor Damien, Pierre-François Gimenez, N. Feyt, V. Nicomette, M. Kaâniche, E. Alata","doi":"10.1109/ISSRE5003.2020.00041","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00041","url":null,"abstract":"Nowadays, air travel is one of the safest transportation means. While safety is historically well integrated into avionics systems, it is becoming increasingly important to take into account the security of such systems for the future. In particular, Host-based Intrusion Detection Systems (HIDS) are commonly used in traditional information systems to improve their security. The adaptation of such systems for deployment inside an aircraft has been studied in another work and has shown to be effective in detecting anomalous behavior in an avionic application. However, the detection itself is not sufficient to provide an on-board reaction, and to prevent such intrusion. This paper proposes to improve such HIDS by introducing a signature-based system capable of providing a first diagnosis after the detection of an anomalous behavior. The proposed diagnosis approach is based on the definition of the signature of an alert, and its comparison with a knowledge database that is regularly updated throughout aircraft lifetime. This approach has been implemented on a real avionic computer and yielded good results in terms of classification accuracy and resources consumption.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117291745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kaushik Madala, Ke Ye Hang, Hyunsook Do, B. Tenbergen
{"title":"SACC - A property driven approach to expose undesired behaviors among system’s components","authors":"Kaushik Madala, Ke Ye Hang, Hyunsook Do, B. Tenbergen","doi":"10.1109/ISSRE5003.2020.00043","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00043","url":null,"abstract":"In recent years, there has been an increase in automation of safety critical systems such as self-driving cars, caretaking robots, or rescue drones. With increase in automation, the risk of systems behaving in an undesired manner has also risen. Most safety analysis approaches predominantly concentrate on identifying or assessing how the failure of a component can affect the behavior of a system based on the system’s requirements. Yet, undesired behaviors can also occur when multiple components concurrently exert opposite effects on a shared resource. To identify safety-critical issues due to undesired concurrent component behaviors, we propose a property-driven approach called safety assessment for concurrent components (SACC). SACC uses a combinatorial technique that considers the requirements specification of a system, expressed as the states and properties of the system’s components for identifying undesired combinations of component behaviors. To evaluate SACC, we performed a study using a requirements document on a caretaking robot. Our results show that SACC identified 38%-80% more undesired system behaviors when compared to the control techniques.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130679011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Designing Neural Networks Using Logical Specs","authors":"Shikhar Singh, Marko Vasić, S. Khurshid","doi":"10.1109/ISSRE5003.2020.00024","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00024","url":null,"abstract":"As systems that deploy machine learning models become more and more pervasive, there is an urgent need to ensure their reliability and safety. While recent years have seen a lot of progress in techniques for verification and validation of machine learning models, reasoning about and explaining their behaviors remains challenging. This paper introduces a new approach for creating machine learning models where instead of the traditional supervised learning using data, the models are directly synthesized from specifications, and thus, are correct by construction. Our focus is binary classifiers with boolean features. Specifically, our approach translates relational specifications written in the well-known modeling language Alloy to neural networks that run on the widely used Tensorflow backend. Our key insight is that a slight enhancement of traditional boolean gates can provide a rich intermediate representation that readily translates to neural networks. To translate the enhanced gates to neural networks, we employ a state-of-the-art program synthesis framework that allows us to find minimal neural networks. The translation of Alloy specifications then follows the standard translation to boolean logic with the exception of utilizing the enhanced boolean gates, followed by a translation to neural networks. We embody our approach in a prototype tool and use it for experimental evaluation. The experimental results show that our approach allows synthesis of neural networks that are hard to create using traditional training methods.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"102 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116345299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exposing Android Event-Based Races by Selective Branch Instrumentation","authors":"Diyu Wu, Dongjie He, Shiping Chen, Jingling Xue","doi":"10.1109/ISSRE5003.2020.00033","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00033","url":null,"abstract":"Android supports an event dispatching system that reacts to system and user actions by generating events. However, lack of synchronization between events can lead to event-based races in Android apps. Such event-based races are difficult to detect dynamically due to the challenges faced in generating the right events to satisfy the right event-dependent conditional branches, so that their guarded racy statements can be reached. As a result, existing dynamic tools, which try to find and reschedule some race-triggering events heuristically, are often ineffective.We introduce SIEVE, a tool for exposing event-based races in Android apps dynamically by leveraging a new selective branch instrumentation technique. For the conditionals potentially affecting a race (detected, say, by a static tool), SIEVE fixes the true/false outcomes of some of these conditionals based on a systematic branch analysis, which analyzes the satisfiability of all the conditionals guarding the given racy statements and their safeness for instrumentation. By instrumenting certain branches selectively this way, we can not only expose effectively event-based races but also reduce substantially the negative ramifications of instrumentation (e.g., reporting non-existent races and introducing unexpected crashes during dynamic execution). An evaluation of SIEVE with 25 Android apps shows that our tool can expose event-based races more effectively than the state of the art.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121973796","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"FSCS-SIMD: An efficient implementation of Fixed-Size-Candidate-Set adaptive random testing using SIMD instructions","authors":"Muhammad Ashfaq, Rubing Huang, Michael Omari","doi":"10.1109/ISSRE5003.2020.00034","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00034","url":null,"abstract":"The Fixed-Size-Candidate-Set (FSCS) version of Adaptive Random Testing (ART) attempts to enhance the fault detection effectiveness of Random Testing (RT) by generating new test cases that are far away from previously executed test cases. Despite its simplicity and good fault-detection effectiveness, FSCS suffers from a very high time cost mainly due to its Single-Instruction-Single-Data (SISD) mechanism for its distance calculation process. To overcome this drawback, in this paper, we propose a novel and efficient implementation of FSCS, namely Fixed-Sized-Candidate-Set using Single-Instruction-Multiple-Data (FSCS-SIMD), which employs SIMD instruction architecture for simultaneous distance calculations of multiple test cases in a many-to-many style. Compared with the original FSCS, our proposed method loads a batch of multiple test cases from the candidate test case set and executed test case set in one CPU execution cycle. After that, a single distance calculation instruction is given to the whole batch for the calculation of all pairwise distances. We conducted a series of simulations and empirical studies to evaluate testing effectiveness and efficiency of our proposed method against FSCS. Our results show that, on average, FSCS-SIMD reduces test case generation overhead of FSCS up to 90%, while maintaining the comparable fault detection effectiveness.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129359410","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the ISSRE 2020 Program Co-Chairs","authors":"","doi":"10.1109/issre5003.2020.00006","DOIUrl":"https://doi.org/10.1109/issre5003.2020.00006","url":null,"abstract":"","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126359294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jianyi Zhou, Feng Li, Jinhao Dong, Hongyu Zhang, Dan Hao
{"title":"Cost-Effective Testing of a Deep Learning Model through Input Reduction","authors":"Jianyi Zhou, Feng Li, Jinhao Dong, Hongyu Zhang, Dan Hao","doi":"10.1109/ISSRE5003.2020.00035","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00035","url":null,"abstract":"With the increasing adoption of Deep Learning (DL) models in various applications, testing DL models is vitally important. However, testing DL models is costly and expensive, e.g., manual labelling is widely-recognized to be costly. To reduce testing cost, we propose to select only a subset of testing data, which is small but representative enough for a quick estimation of the performance of DL models. Our approach, DeepReduce, adopts a two-phase strategy. At first, our approach selects testing data for the purpose of satisfying testing adequacy. Then, it selects more testing data to approximate the distribution between the whole testing data and the selected data by leveraging relative entropy minimization. We evaluate DeepReduce on four widely-used datasets (with 15 models in total). We find that DeepReduce reduces the whole testing data to 7.5% on average and can reliably estimate the performance of DL models.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126510296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"AFLTurbo: Speed up Path Discovery for Greybox Fuzzing","authors":"Lei Sun, Xumei Li, Haipeng Qu, Xiaoshuai Zhang","doi":"10.1109/ISSRE5003.2020.00017","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00017","url":null,"abstract":"Coverage-based greybox fuzzing (CGF) is a common method utilizing coverage information to guide fuzzing. American Fuzzy Lop (AFL) is one of the most famous CGF fuzzers and has been used to uncover thousands of vulnerabilities in many software. However, AFL has two major drawbacks, which impedes it from boosting path discovery: (1) aggressively growing mutation overhead; (2) ineffective mutation region selection. In this paper, we propose three new approaches to overcome the drawbacks: (1) Interruptible mutation, which uses a hang monitor to avoid unnecessary mutation overhead; (2) Locality-based mutation, which utilizes mutation information in previous rounds to guide fuzzing useful regions in future rounds; (3) Hotspot-aware fuzzing, which exploits a pre-evaluation process to identify metadata and only mutates these regions. We combine these approaches into a tool named AFLTurbo based on AFL 2.52b. Furthermore, the effectiveness of AFLTurbo is evaluated in terms of both path discovery and bug detection on eight programs as well as LAVA-M with state-of-the-art fuzzers. The experimental results manifest that AFLTurbo can find 141%, 101% and 41% more paths, and reveal 14 $times$, 30$times$ and 5 $times$ more bugs than AFL, AFLFast and FairFuzz respectively. Additionally, AFLTurbo discovers 20 vulnerabilities, of which 18 are assigned with CVEs.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126606966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}