FRFusion: Flare removal for nighttime infrared and visible image fusion

Abstract

Infrared and visible image fusion (IVIF) aims to integrate the critical information captured by two different sensors effectively. However, most existing methods are designed for well-illuminated environments and often result in the loss of visible details under low-light conditions. Although some low-light enhancement methods for nighttime IVIF have been proposed, they generally incorporate illumination adjustment modules in a simplistic manner, focusing on enhancing intensity information while neglecting the influence of flare artifacts during the enhancement process. To address this limitation, we propose a fusion network called flare removal for nighttime infrared and visible image fusion (FRFusion), which generates flare masks to prevent the loss of complementary information. Specifically, we first design a lightweight multi-scale fusion block (MSFB). In this block, a depthwise separable convolution module (DSCM) combined with a dynamic feature modulation mechanism is employed for efficient local feature extraction. Subsequently, global feature refinement is achieved through an adaptive Fourier filter (AFF) based on the Fourier transform. Moreover, a pretrained auxiliary flare detector (AFD) is used to generate flare masks for constructing a flare-aware fusion loss, which guides the network to suppress flare interference in the fused results. Extensive experiments demonstrate that FRFusion outperforms state-of-the-art (SOTA) methods in both visual quality and quantitative evaluations. In particular, it shows remarkable effectiveness in flare suppression, delivering higher-quality information representation in the fused images.