Contrastive Prototype Clustering for Multimodal Remote Sensing Data Based on Spectral–Spatial Cross Mamba

Abstract

With the increasing diversity of remote sensing (RS) data sources, joint clustering of multimodal RS data demonstrates tremendous potential in Earth observation applications by aggregating multisource information without relying on labeled data. Although significant progress has been made in multiview subspace clustering, existing algorithms still face two limitations: inadequate exploration of complex cross-modal interactions and long-range dependencies, as well as limited capability in handling large-scale multimodal RS datasets. To address these challenges, this article proposes contrastive prototype clustering for multimodal RS data based on spectral–spatial cross Mamba (CPCM). The proposed method encompasses two core innovations. First, we design a multimodal spectral–spatial cross Mamba (S2CM) that performs global contextual modeling with linear complexity in both spectral and spatial dimensions through dual-path Mamba blocks, while employing cross-attention mechanisms to achieve deep semantic fusion of multidimensional features. Second, an end-to-end joint optimization framework is developed, which integrates contrastive learning with clustering learning through a unified objective function. This framework achieves collaborative convergence of feature learning and cluster refinement through an online clustering mechanism that utilizes prototype learning, making it scalable for large-scale multimodal datasets. The effectiveness of the proposed CPCM method is evaluated on three real-world RS datasets: Trento, MUUFL, and Augsburg. Experimental results demonstrate that CPCM achieves overall clustering accuracies of 94.69%, 69.29%, and 83.21% on these datasets, respectively, indicating its superior performance and strong capability in handling large-scale datasets.