Mamba-Wavelet Cross-Modal Fusion Network With Graph Pooling for Hyperspectral and LiDAR Data Joint Classification

Abstract

Recently, with the rapid development of deep learning, the collaborative classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) image has become a research hotspot in remote sensing (RS) technology. However, existing methods either only consider complementary learning of spatial-domain information or do not take into account the intrinsic dependencies between pixels and overlook the importance difference of pixels. In this letter, we propose a mamba-wavelet cross-modal fusion network with graph pooling (MW-CMFNet) for HSI and LiDAR joint classification. First, a two-branch feature extraction (TBFE) is used to extract spatial and spectral features. Then, in order to dig deeper into the complementary information of different modalities and fully fuse them under the guidance of frequency-domain information, a mamba-wavelet cross-modal feature fusion (MW-CMFF) module is devised, it aims to utilize mamba’s outstanding long-range modeling ability to learn complementary information in the spatial and frequency domains, Finally, the graph pooling module is designed to sense the intrinsic dependencies of neighboring pixels and explore the importance difference of pixels, rather than assigning the same weight to different pixels. Experiments on the Houston2013 and Trento datasets show that the MW-CMFNet achieves higher classification accuracy compared to other state-of-the-art methods.