Dynamic 3D-Obstacles Detection by a Monocular Camera and a 3D Map

Abstract

We developed a new method for 3D-obstacles de-tection using a 3D map. Three-dimensional-obstacles detection is a key function of autonomous driving. It is easy to detect static obstacles because they exist in the 3D map. However, the 3D detection of dynamic obstacles that are not in the 3D map is difficult for a typical in-vehicle camera that cannot measure the distance. We aim to detect dynamic obstacles three-dimensionally, using an in-vehicle camera. And we deal with the new problem of accurate 3D reconstruction by using a monocular camera and a 3D map. To solve this problem, we focused on semantic segmentation for detection and depth completion to complement the depth map. We propose a multitask neural network (NN) that shares the encoder of semantic segmentation NN and depth completion NN, whose inputs are an image and a 3D map. The proposed multi-task NN detects dynamic obstacles 1.4 times more accurately than the singletask state-of-the-art method.