Fusion of Millimeter-Wave Radar and Camera Vision for Pedestrian Tracking

Abstract

A fusion of the Millimeter-Wave (MMW) radar and camera vision is proposed for pedestrian tracking in this paper. In this method, the targets of sensors detected are unified onto the polar coordinates. The unscented Kalman filter (UKF) is applied to filter the target information detected by the mmWave radar after the radar signal processing and clustering. Targets image from the camera are detected and localized by YOLOv5 and DeepSORT. The range and velocity detected by camera module are given according to the detected bounding boxes and the projection model. The visual trajectory is then performed by the extended Kalman filter (EKF). Finally, the fusion method of matching targets from radar and camera is given. The simulation results and real experiment result show that the proposed fusion method achieve higher accuracy than individual sensors.