Testing Object Detection for Autonomous Driving Systems via 3D Reconstruction

Abstract

Object detection is to identify objects from images. In autonomous driving systems, object detection serves as an intermediate module, which is used as the input of autonomous decisions for vehicles. That is, the accuracy of autonomous decisions relies on the object detection. The state-of-the-art object detection modules are designed based on Deep Neural Networks (DNNs). It is difficult to employ white-box testing on DNNs since the output of a single neuron is inexplicable. Existing work conducted metamorphic testing for object detection via image synthesis: the detected object in the original image should be detected in the new synthetic image. However, a synthetic image may not look real from humans' perspective. Even the object detection module fails in detecting such synthetic image, the failure may not reflect the ability of object detection. In this paper, we propose an automatic approach to testing object detection via 3D reconstruction of vehicles in real photos. The 3D reconstruction is developed via vanishing point estimation in photos and heuristic based image insertion. Our approach adds new objects to blank spaces in photos to synthesize images. For example, a new vehicle can be added to a photo of a road and vehicles. In this approach, the output synthetic images are expected to be more natural-looking than randomly synthesizing images. The experiment is conducting on 500 driving photos from the Apollo autonomous driving dataset.