Registration-aware cross-modal interaction network for optical and SAR images

Abstract

The registration of optical and synthetic aperture radar (SAR) images is valuable for exploration due to the inherent complementarity of optical and SAR imagery. However, the substantial radiation and geometric differences between the two modalities present a major obstacle to image registration. Specifically, images from optical and SAR require integration of precise local features and registration-aware global features, and features within and across modalities need to be interacted with efficiently to achieve accurate registration. To tackle this problem, we build a Robust Quadratic Net (RQ-Net) based on the paradigm of describe-then-detect, which is of dual-encoder–decoder design, the first encoder is responsible for encoding local features within each modality through vanilla convolutional operators, while the other is an elaborated Multilayer Cross-modal Registration-aware (MCR) encoder specialized in building global relationships both inner- and inter-modalities, which is conducted effectively at various scales to extract informative features for registration. Furthermore, to cooperate with the network’s training for more well-suited registration feature descriptors, we propose a reconsider loss to review whether the least similar positive feature pairs are matchable and make the RQ-Net achieve a higher matching capability. Through extensive qualitative and quantitative experiments on three paired optical and SAR datasets, RQ-Net has been validated as superior in extracting sufficient features for matching and improving image success registration rates while maintaining low registration errors.