Black-Box Testing for Security-Informed Safety of Automated Driving Systems

Abstract

An evaluation of safety and security properties performed by an independent organisation can be an important step towards establishing trust in Automated Driving Systems (ADS), bridging the gap between the marketing portrayal and the actual performance of such systems in real operating conditions. However, due to the complexity of an ADS’s behaviour and dangers involved in performing real environment security attacks, we believe assessments that can be performed with a combination of simulation and validation at test facilities is the way forward.In this paper, we outline an approach to derive test suites applicable to generic ADS feature classes, where classes would have similar capabilities and comparable assessment results. The goal is to support black box testing of such feature classes as part of an independent evaluation. By the means of co-simulation of post-attack behaviour and critical scenarios, we derive a representative set of physical certification tests, to gain an understanding of the interplay between safety and security. During the initial tests an ADS is subjected to various attacks and its reactions recorded. These reactions such as reduced functionality, fall back etc., together with relevant scenarios for the class is further analysed to check for safety implications.