A Radar and Monocular Camera-based Fusion Approach for Pedestrian Detection

Since single sensor has its own shortcomings in pedestrian detection, fusion detection of using radar and camera sensor is currently an effective solution for unmanned driving. This paper proposes a new strategy based on the fusion of radar and camera. First, Kalman filter is used to filter out the invalid data and measurement noise in the radar measurement process, and the radar candidate rectangle is generated based on the target distance and the calibration of the radar/camera sensor. For the camera, extract the foreground information in the video frame and use the information about human area and aspect ratio to screen out suitable pedestrian motion rectangle. Finally, the features are extracted from the candidate rectangle fused by radar and camera and the optimized XGBoost classifier is apply to implement pedestrian recognition. The experimental results show that the detection time of pedestrian after fusion is reduced, and the average precision is increased by 19.41%.