Radar M3-Net: Multi-scale, multi-layer, multi-frame network with a large receptive field for 3D object detection

Abstract

4D millimeter-wave radar has demonstrated significant potential for 3D object detection in autonomous driving due to its cost-effectiveness and robustness. However, the inherent sparsity of radar data poses a significant challenge to achieving accurate 3D object detection, as it limits the amount of meaningful information available for feature learning. The key to addressing the performance degradation caused by sparsity lies in expanding the receptive field and enriching feature representations. To this end, we propose a multi-scale, multi-layer, multi-frame network with a large receptive field, named Radar ${\text{M}}^3$-Net. First, we design a multi-scale voxel feature encoding (MSVFE) module and a multi-layer attention (MLA) module, both of which significantly expand the receptive field and enrich features, effectively addressing the issue of sparsity. Then, a multi-frame fusion module is developed to further enhance features by utilizing the accumulation of temporal information. Simultaneously, we design a novel sparse dual-head within the sparse framework to address the decline in detection accuracy for large object categories caused by radar sparsity. Extensive experiments on the View-of-Delft and TJ4DRadSet datasets have confirmed the advancement and effectiveness of our network. Specifically, our method achieves state-of-the-art mean average precision (mAP) performance on both datasets, even outperforming some multimodal approaches in certain metrics.