Rotation-Robust Remote Sensing Image Classification Method Based on Paired Image Description of Vision-Language Model

Visual-language models, for example, the contrastive language-image pretraining (CLIP), have shown promising results for image classification. However, existing CLIP-based methods fail to describe the fine-grained rotational relationships of paired images and cannot effectively align rotation-associated features in image-text space, limiting their performance in rotation-robust image classification. To address this challenge, we propose a rotation-robust CLIP (RoRoCLIP) model for remote sensing image classification. RoRoCLIP contains two key components, a dual image feature extraction (DIFE) module and a rotation awareness (RoA) module. The DIFE module extracts features from both the original and rotated images via the pretrained encoder of CLIP, and performs image-level feature interactions via a learnable transformer layer. The RoA module associates the textual prompt ‘differences caused by rotation’ with differential visual features extracted by DIFE, and aligns rotation-associated features in image-text space. Based on these two modules, we construct a classification loss and an RoA loss to optimise the model, enabling RoRoCLIP to perceive rotational variations and learn discriminative features for image classification. Experimental results on the NWPU-VHR-10 and RSOD remote sensing datasets demonstrate that the proposed model enhances the CLIP's robustness against image rotation, and outperforms the state-of-the-art approaches in classification accuracy.