CRRFNet: An adaptive traffic object detection method based on camera and radar radio frequency fusion

Abstract

A large number of studies have proved that camera and radar fusion is a useful and economical solution for traffic object detection. However, how to improve the reliability and robustness of fusion methods is still a huge challenge. In this paper, an adaptive traffic object detection method based on a camera and radar radio frequency Network (CRRFNet) is proposed, to solve the problem of robust and reliable traffic object detection in noisy or abnormal scenes. Firstly, two different deep convolution modules are designed for extracting features from the camera and radar; Secondly, the camera and radar features are catenated, and a deconvolution module is built for upsampling; Thirdly, the heatmap module is used to compress redundant channels. Finally, the objects in the Field of View (FoV) are predicted by location-based Non-Maximum Suppression (L-NMS). In addition, a data scrambling technique is proposed to alleviate the problem of overfitting to a single sensor by the fusion method. The existing Washington University Camera Radar (CRUW) dataset is improved and a new dataset named Camera-Radar Nanjing University of Science and Technology Version 1.0 (CRNJUST-v1.0) is collected to verify the proposed method. Experiments show that CRRFNet can detect objects by using the information of radar and camera at the same time, which is far more accurate than a single sensor method. Combined with the proposed data scrambling technology, CRRFNet shows excellent robustness that can effectively detect objects in the case of interference or single sensor failure.