Enhanced RSVQA Insight Through Synergistic Visual-Linguistic Attention Models

Abstract

The interpretation of remote sensing images remains a significant challenge due to their complex, information-rich nature. Current remote sensing visual question answering (RSVQA) techniques have been a step forward toward building intelligent analysis systems for remote sensing images. However, most existing RSVQA models that rely on ResNet, VGG, and Swin transformers as visual feature extractors often fail to capture complex visual relationships, particularly the intricate dependencies between segmented regions and depth-related features in remote sensing data. To address these limitations, this letter introduces a novel RSVQA approach that leverages state-of-the-art components with an innovative architecture to advance interactive remote sensing analysis. The proposed model features a novel dual-layer visual attention mechanism in the representation module to process intricate features and capture regional relationships alongside processing the overall features. The fusion module employs a unique attention-based design, combining both self-attention and mutual attention, to integrate these features into a unified vector representation. Finally, the answering module utilizes a refined multilayer perceptron classifier for precise response generation. Evaluations on an RSVQA benchmark demonstrate the system’s superiority over existing methods, marking a significant step forward in remote sensing analytics.