FRFusion: A deep fusion framework for infrared and visible images based on fast fourier transform and retinex model

Abstract

Infrared and visible image fusion exploits the complementary characteristics of both modalities to produce a richer and more visually enhanced image. However, most existing methods primarily focus on well-lit conditions and tend to overlook texture and contrast degradation in low-light environments. Furthermore, these approaches often neglect frequency domain information during feature extraction. We propose a network called FRFusion for fusing infrared and visible images in low-light environments to address the aforementioned challenges. Firstly, based on the Retinex model, we designed encoders with different structures to decompose visible images into reflection and illumination components. During this process, we introduced a feature adjustment module ($FAM$) to enable the model to simultaneously extract information from the input image in spatial and frequency domains. It is worth noting that the extraction of infrared features pertains to the encoder structure of the reflection components of visible images. Secondly, in the feature fusion stage, we introduced the dual attention feature fusion module ($DAFFM$) to fully integrate the global and local features of infrared and visible images, thereby achieving a more comprehensive synthesis of complementary information. Finally, we propose a brightness adaptive network ($BAN$) for the illumination component, which restores the brightness information of the fused image by adaptively adjusting the brightness features. Experimental results on three public datasets demonstrate that our method excels in both visual quality and evaluation metrics.