CM2-STNet: Cross-modal image matching with modal-adaptive feature modulation and sparse transformer fusion

Abstract

Multimodal image matching is a fundamental task in geospatial analysis, aiming to establish accurate correspondences between images captured by heterogeneous imaging devices. However, significant geometric inconsistencies and nonlinear radiometric distortions lead to large distribution gaps, posing a major challenge for cross-modal matching. Moreover, existing methods often struggle to adaptively capture intra- and inter-modal features at multiple scales and to focus on semantically relevant regions in large-scale scenes. To address these issues, we propose a novel cross-modal image matching network called CM²-STNet. Specifically, we introduce a modal-adaptive feature modulation (MAFM) module that dynamically adjusts cross-modal feature representations at multiple scales, thereby enhancing semantic consistency between modalities. In addition, a cross-modal sparse transformer fusion (CM-STF) module is developed to guide the network to concentrate on the most relevant regions, where a Top-k selection mechanism is employed to retain discriminative features while filtering out irrelevant content. Extensive experiments on multimodal remote sensing datasets demonstrate that CM²-STNet achieves accurate and robust matching performance, validating its effectiveness and generalization ability in complex real-world scenarios. Code and pre-trained model are available at https://github.com/whuzzzz/CM²-STNet.