SFDFNet: Leveraging spatial-frequency deep fusion for RGB-T semantic segmentation

Due to the insensitivity to lighting variations, the RGB-Thermal (RGB-T) semantic segmentation models show significant potential in processing images captured under adverse conditions, such as low light and overexposure. Current RGB-T semantic segmentation methods usually rely on complex spatial domain fusion strategies, yet they neglect the complementary frequency characteristics of RGB and thermal modalities. Through frequency analysis, we find that thermal images focus on low-frequency information, while RGB images are rich in high-frequency details. Leveraging these complementary properties, we introduce the Spatial-Frequency Deep Fusion Network (SFDFNet), which employs a dual-stream architecture to enhance RGB-T semantic segmentation. Key innovations include the Distinctive Feature Enhancement Module (DFEM) to improve feature representation in both modalities and the Spatial-Frequency Fusion Module (SFFM), which integrates spatial and frequency features to optimize cross-modal fusion. Extensive experiments on three RGB-T datasets demonstrate the superior performance of our method, both qualitatively and quantitatively, compared to state-of-the-art models.