Pedestrian Detection by Fusion of RGB and Infrared Images in Low-Light Environment

Abstract

Pedestrian detection in low-light environment is an essential part for autonomous driving in all-day and all-weather situations. A current trend is utilizing multispectral information such as RGB and infrared images to detect pedestrians. Despite its efficacy, such an approach suffers from underperformance in dealing with varied object scales due to its limited feature fusion on semantic levels. To address the above problem, we propose a novel multi-layer fusion network called as MLF-FRCNN. In this network, multi-scale feature maps are created from RGB and infrared channels from each backbone block. A feature pyramid network module is further introduced to facilitate predictions on multi-layer feature maps. The experimental results on the KAIST Dataset reveal that our method achieves a runtime performance of 0.14s per frame and an average precision of 91.2% which outperforms state-of-the-art multispectral fusion methods. The effectiveness of our approach in dealing with scaled objects in low-light environment is further proven by ablation studies.