Research on Multimodal Fusion Perception Technology for Autonomous Sweeping Vehicle

This article proposes a multimodal fusion framework to address the challenges of detecting and tracking specialized vehicles and dynamic targets in complex industrial park environments. The framework integrates LiDAR, a monocular Camera, and an inertial navigation system (INS) to achieve precise obstacle perception and stable tracking through dynamic region of interest (ROI) cropping, optimized point cloud clustering, target detection, and multimodal perception fusion. First, a path-aware dynamic ROI cropping method and a multiregion density-aware seed point cloud ground segmentation approach are introduced to improve adaptability and point cloud processing efficiency. Second, a two-stage refinement strategy method is proposed to enhance target clustering accuracy. Furthermore, by combining the 2-D detection network, a multimodal perception fusion module, and a multiobject tracking (MOT) strategy, the framework significantly improves fusion efficiency and matching accuracy. Field tests demonstrate that the framework achieves excellent performance, with static object localization deviations below 0.8 m and reliable state estimation for dynamic targets. On a custom dataset, the monocular Camera achieves 91.67% accuracy for specialized vehicles, while the fusion framework exhibits strong adaptability and reliability in complex scenarios.