A versatile interactive dual-path architecture with text modulation for multi-band image fusion

Abstract

Image fusion aims to integrate complementary information from source images to generate images with more comprehensive detail representation. Compared to conventional single-band images, multi-band images provide a wider array of radiative details and information. With the rapid advancement of deep learning techniques, the fusion of multi-band images can provide enhanced feature representations for target detection, demonstrating significant application value across military, medical, and environmental monitoring domains. Despite its growing popularity, image fusion remains a challenging problem due to inherent discrepancies in how different sources depict scene content. Current methods generally suffer from three limitations: 1) Sensitivity to slight misalignment between source images, leading to artifact generation in fused results; 2) Ineffective handling of interference caused by low quality source images, such as noise or degradation, etc. And 3) lack of interactive mechanisms to accommodate diverse subjective and objective requirements. To address these challenges, we propose a versatile interactive dual-path architecture with text modulation for multi-band image fusion. First, a micro registration deformation dual-path fusion module is designed, which employs explicit deformation fields to compensate for geometric misalignment, thereby mitigating artifacts, while incorporating a feature adaptive selection mechanism to enhance texture details and contrast. Second, we proposed a dynamic text modulated fusion module utilizing a dual-path attention mechanism, where text embeddings serve as conditional signals to drive both channel-wise and spatial attention weight generation, simultaneously addressing image quality degradation and interactive flexible fusion requirements. Extensive experiments conducted on two public benchmark datasets and one self-constructed multi-band infrared dataset prove that our method is superior to state-of-the-art (SOTA) methods in terms of quantitative evaluations and qualitative evaluations, effectively enhancing image fusion performance and degradation treatment.