Object Detection Algorithm for Railway Scenes Based on Infrared and RGB Image Fusion

Abstract

The driver-assistance system tends to fuse multi-modal sensor data, for instance, the infrared and RGB sensors, to detect intrusion objects to enhance driving safety. However, the semantic misalignment dilemma and the spectral imb-alance between infrared and RGB images make it hard to exp-loit the advantages of multi-sensors in the end-to-end learning system. To solve these problems, we employ the widely used affine transformation on our railway dataset to solve the se-mantic-misalignment issue, in addition, we propose a fusion module, DMF, to fuse the well-aligned features, which can bri-dge the domain gap among different sensors. To this end, we propose an efficient railway invasive object detection network, YOLOv5s-DMF. Compared with the state-of-the-art metho-ds, the YOLOv5s-DMF substantially reduces the MR by 14.23% by employing the well-established decouple head. And our YOLOv5s-DMF further increases the mAP@0.5 by 5.7% and the mAP@0.5:0.95by4.1%.