Assuring the Safety of End-to-End Learning-Based Autonomous Driving through Runtime Monitoring

Artificial intelligence is a promising element in the development of autonomous vehicles. We have designed an end-to-end learning-based autonomous driving system solely with a neural network through supervised learning, which has been deployed on a model vehicle equipped with a lidar. Input of the convolutional neural network are the point clouds from the lidar and outputs are the requested drive torques and steering angles. For the training of the neural network, the required sensor and actuator data were recorded by remotely controlling the model vehicle. With supervised learning, the end-to-end neural network learns the safety-relevant rules only implicitly through examples in the training data. Because it is not guaranteed that the neural network has learned all necessary rules and can apply them correctly in all situations, safe operation cannot be assured. To address this safety issue, we developed a software architecture including a runtime monitoring component. If the runtime monitoring component detects a violation of any predefined safety rule, it will select an appropriate strategy in order to transfer the vehicle into a safe state.