DPSFusion: Dynamic Pseudo-Supervision and Semantic Guidance for Multi-Modality Image Fusion

Abstract

Existing multimodal image fusion methods mainly emphasize network design and fusion rules to combine complementary information from different modalities. However, the supervision information adopted during training is often not aligned with the evaluation metrics employed to assess fused images. As a result, this misalignment biases optimization toward pixel similarity rather than perceptual quality and detail preservation, which causes blurred edges, texture loss, and modality imbalance. In addition, when task labels are unavailable, many approaches fail to fully exploit semantic relations between modalities. This weak alignment of cross-modal features leads to missing salient regions. To address these issues, we propose DPSFusion, a novel fusion framework that integrates a dynamic pseudo-supervision (DP) mechanism and a semantic guidance (SG) module. Specifically, the DP mechanism constructs pseudo-supervision images from diverse candidate fused results and adaptively identifies high-quality References using a multi-metric Fusion Quality Score (FQS). The pseudo-supervision set is subsequently iteratively updated during training, which ensures that the optimization objective gradually aligns with the evaluation metrics employed to assess fusion quality. In parallel, SG module is designed to exploit semantics without task-specific labels in order to produce fused images with sharper structures and higher target saliency. It extracts multi-level embeddings from infrared, visible, and pseudo-supervision images using a Contrastive Language-Image Pre-training (CLIP) model and converts them into feature modulation inside the network to highlight modality strengths and suppress redundancy. Finally, experiments on MSRS, RoadScene, TNO, and M³FD show state-of-the-art results on six standard fusion metrics and improved downstream performance in semantic segmentation and object detection, demonstrating the effectiveness of DPSFusion.