OSHFNet: A heterogeneous dual-branch dynamic fusion network of optical and SAR images for land use classification

Optical and synthetic aperture radar (SAR) images are two of the most widely used remote sensing data sources, providing complementary but structurally consistent information. This complementarity has inspired significant research on their fusion. However, due to the huge difference in image representation between optical and SAR data, this difference will lead to inaccurate information expression when using the same structure to extract features, resulting in poor performance in classification tasks. Therefore, in the feature extraction stage, we analyze the respective advantageous features of optical and SAR images and propose a heterogeneous dual-branch network framework. Our framework exploits the rich local features of optical images and the global structural features of SAR images by using CNN and VMamba as their respective feature extractors. This heterogeneous feature extraction strategy effectively captures the complementary features of different modalities and provides a solid foundation for subsequent feature fusion. Second, in the feature fusion stage, we introduce a global-local dynamic gating fusion module. The use of multi-scale feature extraction and self-attention mechanism ensures comprehensive feature capture, while the dynamic gating mechanism enhances the integration of cross-modal complementary information. Finally, our method achieves excellent performance on medium and high-resolution datasets, showing robustness and adaptability at different resolutions. Notably, it significantly improves the overall classification accuracy while maintaining the accuracy of individual categories. For challenging categories such as roads, our method achieves significant improvements of about 15%.