LiDAR-camera fusion: dual-scale correction for vehicle multi-object detection and trajectory extraction

Abstract

The different principles of sensor technology determine their distinct performance in vehicle detection and microscopic tracking. Vision-based sensors can provide rich semantic information but lack reliable spatial information, and their reliability is reduced in complex lighting conditions. On the other hand, LiDAR can offer high-precision spatial information independent of lighting conditions, but it suffers from low resolution and effective sampling rate limitations. Considering the strong complementarity between images and point clouds, efficient object detection can be achieved by leveraging their synergy. However, existing research has not fully explored the correlation between the features of these two types of data. This paper proposes a novel dual-scale correction strategy for feature-level fusion of camera and LiDAR data. This strategy captures spatial features of point clouds and semantic features of images at both global and local scales and establishes mapping relationships separately. The global correction results are iteratively updated based on the results of local precision correction. To validate the effectiveness of the proposed method, data is collected from highway and urban expressway scenarios. The results indicate improvements in both object detection and microscopic trajectory tracking performance compared to using single sensors alone. Furthermore, the fusion approach outperforms other methods in terms of detection accuracy and processing time. This research offers a method for real-time and accurate extraction of vehicle trajectories using roadside cameras and LiDAR devices, with potential applications in real-time trajectory tracking and traffic management.