Enhanced visible light detail and infrared thermal radiation for dual-mode imaging system via multi-information interaction

Abstract

In the field of dual-mode optical imaging, image fusion techniques offer significant advantages such as improved spatial resolution and the suppression of redundant information, particularly for visible and infrared image. However, existing fusion methods often overlook the interaction between multiple feature information during the extraction and fusion stages, resulting in the inability to extract both visible light detail and infrared thermal radiation effectively. To address this challenge, we construct a dual-mode imaging system and propose an image fusion method that incorporates Convolution-Swin-Transformer Blocks (CSTBs). The block combines Convolution and Shifted Window Transformer to improve the interaction and extraction between local and global information within the images. On the other hand, our proposed method strengthens the comprehensive interaction and fusion between shallow pixel-level information and deeper semantic representation by fusing local and global feature information at various layers. Furthermore, we introduce a multi-component loss function that balances the complementary features extracted from the source images, with a particular focus on enhancing edge texture, structure, and brightness information. Experimental results demonstrate that our method achieves superior performance in simultaneously enhancing both texture details and thermal radiation. This is evidenced by results on two publicly available datasets, as well as the Target_GDUT dataset captured using our dual-mode optical imaging system.