Multi-modal 3D multi-object tracking with robust association and track drift compensation

3D multi-object tracking is crucial for enhancing the understanding of the environment in autonomous driving and robotics. Low-quality detections and less robust associations are two challenges in the point-aware tracking-by-detection paradigm. Conventional approaches suffer from inadequate pre-processing of detected outliers, and poor appearance-based associations during occlusion. To address these issues, this paper proposes a real-time and robust 3D multi-object tracking framework based on the fusion of camera and LiDAR data. Firstly, a two-level association strategy is introduced, whereby high-confidence tracks and detections are initially linked through a straightforward 3D IoU cost, followed by the association of remaining entities using discriminative deep appearance features, emphasizing the similarity between the recently updated track appearance and reemerging targets within dynamically constrained search boundaries. Secondly, a track drift compensation method is presented to refine the low-quality detections using their historically matched tracks, facilitating accurate updates accordingly. Experiments show that the proposed method achieved 79.36 % HOTA and 74 % AMOTA in KITTI and nuScenes benchmarks, respectively. This result surpasses many advanced solutions, particularly exhibiting robust performance in occluded environments.