A Geometric Convolutional Neural Network for 3D Object Detection

Abstract

We propose a method for accurate 3D vehicle detection based on geometric deep neural networks. From only a single RGB image, the framework is able to recover the 3D positions and predict 3D bounding boxes. In particular, the algorithm leverages single image depth estimation and semantic segmentation to produce 3D point cloud for specific objects. By geometrically constraining the object dimensions, an accurate and stable 3D bounding box which tightly fits into the real object can be estimated. We verify the effectiveness and robustness of our method by comparing with other recent state-of-art methods on the challenging KITTI 3D benchmark dataset as well as synthetic Virtual KITTI dataset. Without requiring ground truth 3D labels, our method is able to produce competitive and robust performance in 3D scene understanding and detection.