Multimodal feature adaptive fusion for anchor-free 3D object detection

LiDAR and camera are two key sensors that provide mutually complementary information for 3D detection in autonomous driving. Existing multimodal detection methods often decorate the original point cloud data with camera features to complete the detection, ignoring the mutual fusion between camera features and point cloud features. In addition, ground points scanned by LiDAR in natural scenes usually interfere significantly with the detection results, and existing methods fail to address this problem effectively. We present a simple yet efficient anchor-free 3D object detection, which can better adapt to complex scenes through the adaptive fusion of multimodal features. First, we propose a fully convolutional bird’s-eye view reconstruction module to sense ground map geometry changes, for improving the interference of ground points on detection results. Second, a multimodal feature adaptive fusion module with local awareness is designed to improve the mutual fusion of camera and point cloud features. Finally, we introduce a scale-aware mini feature pyramid networks (Mini-FPN) that can directly regress 3D bounding boxes from the augmented dense feature maps, boosting the network’s ability to detect scale-varying objects, and we additionally construct a scene-adaptive single-stage 3D detector in an anchor-free manner. Extensive experiments on the KITTI and nuScenes datasets validate our method’s competitive performance.