Cross-Modality Target Detection Using Infrared and Visible Image Fusion for Robust Objection recognition

Abstract

Visible-infrared cross-modal object detection aims to overcome the limitations of single modality highlighting in complex environments (rain, fog, weak light) by utilizing dual-modal images. Most existing methods typically use finite size convolution kernels to learn local features, and ignore the interaction of non-local feature dependencies between modalities such as the infrared and the visible modalities, resulting in unsatisfactory detection performance. To tackle the problem, we propose a multi-modal object detection algorithm that fuse visible and infrared modalities through cross enhancement and long-range guidance, effectively combining complementary information and shared collaborative information to enhance detection capabilities. In this paper, we first propose the cross-modality feature enhancement method that utilizes the difference between channel information and spatial information of each modality. Secondly, we use cross-attention layers on the basis of transformer to achieve long-range interactive information exchange, and add self-attention layers to enhance internal connections. Finally, we propose a feature enhancement module that enhances performance by utilizing a multi-branch structure composed of different convolutions. Experiments on three publicly available datasets have shown that our proposed approach achieves superior robustness and accuracy under all weather conditions and constantly changing lighting conditions.