SCMFusion: Semantic Constrained Multi-Scale Fusion Network for infrared and visible image fusion

Due to the distinct characteristics of infrared and visible images, we introduce Semantic Constrained Multi-Scale Fusion (SCMFusion) to balance unique and common features during infrared and visible image fusion (IVIF). This method reduces redundancy and comprehensively represents scenes captured by both modalities. Firstly, the semantic-constrained Frequency-Aware Bidirectional Pyramid (FABP) combines a spatial pyramid, which vertically keeps the channel unchanged and captures a larger receptive field through resolution reduction, with a channel pyramid, which preserves scale consistency and enriches feature expression through increased channels. Subsequently, the extracted features undergo Semantic-Constrained Cross-Modal Cross-Scale Fusion (SC-CCF) for effective information exchange and fusion. Next, the semantic constraints ensure pixel-wise alignment between fused features and original images, integrating modality-specific features and enhancing shared features. Finally, a Reconstruction Block (RB) processes high- and low-frequency components to produce the fused image. Comparative experiments demonstrate that our model outperforms 11 state-of-the-art (SOTA) fusion methods and achieves notable results in object detection.