Alternating interaction fusion of Image-Point cloud for Multi-Modal 3D object detection

Abstract

A mainstream feature fusion method involves enhancing Lidar point cloud information by incorporating camera, but it fails to fully utilize the rich information in images. Another method uses a dual-channel parallel approach to fuse image and point cloud information, but it also faces issues such as excessive module stacking and high computational demands. Therefore, we propose a powerful alternating interaction fusion approach. Firstly, it resolves the problem of unilateral fusion schemes that overly rely on point cloud information and fail to fully utilize image data. Secondly, it tackles the problem of excessive module stacking and high computational demands in dual-channel parallel fusion schemes of point cloud and image data. Specifically, our alternate interactive fusion module implements a method where image and point cloud BEV features mutually enhance each other. Local attention interactions are engaged between image features containing point cloud information and regular image features. This enhances the expressiveness of image features. Subsequently, internal BEV attention interactions occur between point cloud BEV features with enriched image information and regular point cloud BEV features. This step improves the expressiveness of the point cloud BEV features. Experiments on the large-scale nuScenes dataset demonstrate that our proposed method outperforms both the unilateral point cloud-centric fusion and the parallel interactive fusion approaches.