CPIGAN: Infrared and visible image fusion via cross-scale progressive interaction network with adversarial learning

Abstract

The objective of infrared and visible image fusion is to synthesize a single fused image that retains the salient target features and texture details of the source image. However, existing image fusion algorithms have not yet fully considered the intrinsic depth characteristics of images, ignoring the correlation between their information at different scales, thus limiting the fusion performance. Toward this end, we propose a cross-scale progressively interacting adversarial fusion network, called CPIGAN. In particular, in the generator, we design a progressively interacting feature extractor, which consists of the dual-stream gradient residual enhancement module (DGREM) and the multimodal cross perception module (MCPM). This design not only achieves feature-level texture enhancement, but also facilitates the full interaction of relevant and complementary information of multimodal images at different scales. Furthermore, we propose a cross-scale cross-fusion strategy that combines global and local attention models. It enables the accurate capture of local details at the spatial level while providing a comprehensive grasp of global information at the channel level. Extensive experiments show that our CPIGAN outperforms other advanced methods in subjective and objective evaluations. Meanwhile, we demonstrate the superiority of our method by evaluating it in the downstream task of object detection.