TCIGFusion: A two-stage correlated feature interactive guided network for infrared and visible image fusion

Abstract

Infrared and visible image fusion is aimed at generating images with prominent targets and texture details, providing support for downstream applications such as object detection. However, most existing deep learning-based fusion methods involve single-stage training and manually designed fusion rules, which cannot effectively extract and fuse features. Therefore, in this paper, we propose a two-stage correlated feature interactive guided network termed TCIGFusion. In the first stage, a Unet-like dual-branch Transformer module and dynamic large kernel convolution block (DLKB) are used to extract global features from the two source images, while the convolution blocks extract local features from the source images. In the second phase, we designed a cross attention guide module (CAGM) to interactively fuse the heterogeneously related features of the two modalities, avoiding the complexity associated with manually designing fusion rules. Furthermore, to optimize the efficacy of the fusion network, we employ a combination of image reconstruction, decomposition, and gradient loss functions for unsupervised training of the model. The superiority of our TCIGFusion is evidenced by extensive experimentation conducted on multiple public datasets. These experiments demonstrate that our method outperforms other state-of-the-art deep learning approaches, as evaluated through both subjective and objective metrics.