LSCANet: Differential features guided long–short cross attention network for infrared and visible image fusion

Abstract

Infrared and visible image fusion can generate images that not only highlight prominent targets, but also contain rich details and texture information. However, directly fusing the features of infrared and visible images can diminish the correlation information between source images. To address this, we propose a differential features guided long–short cross attention network for infrared and visible image fusion (LSCANet). Specifically, a differential feature cross attention network (DCAN) is designed to achieve cross modal multi-scale interaction of differential features in the feature extraction process. Cross modal feature interaction before infrared and visible features fusion can enhance deep feature relationships between cross modal features, thereby preserving more correlation information between source images. Besides, a long–short differential feature attention network (LSDAN) is proposed to achieve the integration of multi-scale cross-modal differential features, which can preserve details and texture information while reducing the artifacts and noise introduced during the integration process. Moreover, the loss function is introduced to impel the network retain more details and texture information while preserving thermal radiation information. Ablation experiments were conducted to validate the effectiveness of LSCANet. Extensive qualitative and quantitative experiments conducted on cross dataset benchmarks have demonstrated that LSCANet outperforms eight state-of-the-art methods. The source code is available at https://github.com/Pinot-30/LSCANet/tree/main.