Camera-LiDAR Fusion Based Three-Stages Data Association Framework for 3D Multi-Object Tracking

Abstract

3D multi-object tracking (MOT) ensures safe and efficient motion planning and vehicle navigation and plays an important role in perception systems in autonomous driving. Currently MOT is divided into tracking by detection and end-to-end, but most of them are tracking by detection using only single depth sensor such as LiDAR to detect and track objects. However, LiDAR has the limitation of not being able to obtain information about the appearance of the object due to the lack of pixel information, which can lead to obtaining inaccurate detection results thus leading to erratic tracking results. Therefore, in this paper, we propose a novel 3D MOT framework that combines the unique detection advantages of cameras and LiDAR. To avoid the IDs generated by the early death of the detection of the same object that produced low scores in successive frames, we design a 3D MOT framework with three-stages data association. And we also design a data association metric based on 3D IoU and Mahalanobis distance. The camera-LiDAR fusion-based 3D MOT framework we propose proves its superiority and flexibility by quantitative experiments and ablation experiments.