Enhanced association and detection accuracy in multi-object tracking with camera-LiDAR Data Fusion

Traditional multiple-object tracking methods employ single sensors, such as optical LiDAR or cameras. However, using only a single sensing modality may cause information loss, which impairs the tracking performance. Thus, the present study proposes an object tracking system in which 2D and 3D detection boxes acquired by a camera and LiDAR sensor, respectively, are combined and preprocessed before being input to the tracker. The efficiency of the association algorithm used to match the detection boxes with existing target boxes is enhanced by utilizing a simple Intersection over Union (IOU) measure and prioritizing the high-confidence 3D detection boxes in the matching process. Moreover, following the association process, the 2D information is used to update the state of the 3D tracking boxes in the following image frame, thereby improving the stability and continuity of the 3D trajectories and enhancing the tracking performance. The performance of the proposed method is evaluated using the HOTA and sAMOTA metrics. The results confirm that using complementary information obtained from 2D and 3D detectors significantly improves the accuracy and robustness of the object tracking process.