DTEFusion: a dual-transformation network enhanced fusion method for infrared and visible images with improved mamba

Abstract

The primary goal of image fusion is to generate clearer and more visually perceptible results consistent with human vision. Most existing fusion methods primarily focus on optimizing specific objective metrics, which often results in insufficient representation of the inherent multiscale and multidirectional characteristics of human visual perception. To address this issue, a dual-transformation enhancement network (DTEFusion) is more suited to the human visual system. It is combined with an improved Mamba module for infrared and visible image fusion, mimicking the human visual mechanism by jointly modeling multiscale structural details and cross-modal feature interactions to achieve more perceptually consistent fusion results. Specifically, the proposed frequency channel attention nonsubsampled wavelet transform (F-NSWT)module combines frequency-channel attention with a nonsubsampled wavelet transform to enhance multiscale frequency-domain features in visible images; the FreMLP module strengthens target feature representations in infrared images; and the token-convolution mamba (TCM) module integrates the advantages of CNN and Mamba to effectively capture both local and global dependencies while maintaining computational efficiency. In addition, the cross-model skip-dense connection (C-SDC) fusion layer enables adaptive cross-modal feature interaction. Moreover, we employ a particle swarm optimization (PSO) algorithm to tune the loss function hyperparameters and adaptively optimize our objective metrics. Experimental results demonstrate that our method achieves superior overall performance compared with eleven state-of-the-art fusion algorithms. Our approach attains the highest MS-SSIM scores across four benchmark datasets, indicating closer alignment with human visual perception. The code for this paper will be released at https://github.com/bluemountain123 /DTEFusion.