TSH-FCNet: Triple-source heterogeneous remote sensing images fusion classification network based on feature propagation and perception

With the diversification of remote sensing (RS) sensor types, the accessibility and availability of various RS data types are continuously improving. The collaborative use of multi-source RS data can comprehensively and effectively improve the accuracy of RS for earth observation. However, current research on multi-source RS image fusion classification primarily focuses on only two types of RS data. The heterogeneous characteristics of three or more types of RS data significantly complicate the data fusion process. In particular, how to effectively explore the correlations among the inherent characteristics of three or more heterogeneous RS data remains a critical challenge that has not been effectively addressed. This greatly affects the accuracy of RS land classification and other earth observation tasks. To address this issue, a TSH-FCNet based on feature propagation and perception for collaborative classification of hyperspectral (HS), multispectral (MS), and radar images is proposed. This network thoroughly explores the intrinsic correlations among the three heterogeneous data sources and employs an innovative feature interaction mechanism to leverage their complementary advantages. It overcomes the interference of heterogeneous characteristics between different data sources on fusion, effectively enhancing the final classification accuracy. Specifically, a distance similarity attention guides the mutual perception and fusion of triple-source RS information, promoting the flow of complementary features among the triple-source and improving the final classification accuracy. Additionally, the shared information from the triple-source RS data is injected into the features to be fused through a domain alignment mechanism, enhancing the spatial and semantic consistency of the features, thereby strengthening the classification model’s ability to recognize complex surface features. We tested the algorithm on three triple-source RS datasets. The experimental results indicate that the proposed algorithm achieves significant improvements over existing mainstream methods, exhibiting greater stability and reliability when handling highly heterogeneous and diverse data sources. The implementation code of this algorithm will be available from https://github.com/cwlnnu/TSH-FCNet.