Improving the Robustness of Automotive Gesture Recognition by Diversified Simulation Datasets

Abstract

A key element for the interaction between pedestrians and autonomous vehicles is the automated recognition of traffic and communication gestures. Gestures help vehicles to resolve critical or ambiguous situations. Detecting gestures with radar sensors is advantageous with respect to environmental conditions and lighting. However, the collection of a radar dataset that covers the wide range of variations in automotive scenarios comes at high cost and effort. On the other side, datasets with limited variations lead to reduced recognition accuracy or even complete failure in new scenarios. Hence, this paper analyzes the impact that deficiencies of traffic gesture datasets can have on the accuracy and investigates mitigation strategies based on the augmentation by simulated, variation-rich radar data. It is shown that by augmentation the robustness of a convolutional neural network (CNN)-based classifier against variations not covered by the training data is significantly improved. As a key result, both complete failure of the classifier and strongly decreased classification accuracy are avoided.