Relational representation learning network for cross-spectral image patch matching

Abstract

Recently, feature relation learning has drawn widespread attention in cross-spectral image patch matching. However, existing related research focuses on capturing diverse feature relations between image patches and ignores sufficient intrinsic feature representations of individual image patches. To address this limitation, we propose an innovative relational representation learning that simultaneously focuses on sufficiently mining the intrinsic features of individual image patches and the feature relations between image patches. Based on this, we construct a Relational Representation Learning Network (RRL-Net). Specifically, we innovatively construct an autoencoder to effectively characterize the individual intrinsic features, and introduce a feature interaction learning (FIL) module to extract deep-level feature relations. Meanwhile, to further fully mine individual intrinsic features, a lightweight multi-dimensional global-to-local attention (MGLA) module is constructed to enhance the global feature extraction of individual image patches and capture local dependencies within global features. By combining the MGLA module, we further explore the feature extraction network and construct an attention-based lightweight feature extraction (ALFE) network. Furthermore, a multi-loss post-pruning (MLPP) optimization strategy is proposed, which can greatly facilitate network optimization while avoiding increases in parameters and inference time. Extensive experiments demonstrate that our RRL-Net achieves state-of-the-art (SOTA) performance on multiple public datasets. Our code is available at https://github.com/YuChuang1205/RRL-Net.