An Efficient Target Recognition Model Based on Radar–Vision Fusion for Road Traffic Safety

It is difficult for automated driving systems, or advanced driver assistance systems, to recognize and comprehend their surroundings. This paper proposes a transformer model-based approach for road object recognition using sensor fusion. Initially, data from the camera and millimeter-wave (mmWave) radar are simultaneously acquired and pre-processed. Since direct point cloud-to-image fusion is difficult for fusion object detection models, the radar point clouds are then circularly projected onto a 2-dimensional (2D) plane. Then, both the camera image and radar projection image enter different branches of the feature extraction model, utilizing a dual-path vision transformer (DualP-ViT) to complete feature extraction and fusion. The items are recognized after going through several layers of encoders and decoders. An encoder decoder-based vision transformer (EDViT) provides accurate measures of distance and velocity. Also, the vision sensors (cameras) produce high-resolution images with rich visual information. The proposed approach is implemented on the nuScenes dataset, and the performance is evaluated based on object detection metrics. The mean Average Precision (mAP), NuScenes Detection Score (NDS), Planning KL-Divergence (PKL), accuracy, precision, recall, f1-score, and latency performance obtained with the proposed approach is 59, 68, 0.6, 80, 79, 80, 78.9, and 10 ms. In the proposed approach, the robustness and accuracy of object detection is improved.