Context-driven and sparse decoding for Remote Sensing Visual Grounding

Abstract

Remote Sensing Visual Grounding (RSVG) is an emerging multimodal RS task that involves grounding textual descriptions to specific objects in remote sensing images. Previous methods often overlook the impact of complex backgrounds and similar geographic entities during feature extraction, which may confuse target features and cause performance bottlenecks. Moreover, remote sensing scenes include extensive surface information, much of which contributes little to the reasoning of the target object. This redundancy not only increases the computational burden but also impairs decoding efficiency. To this end, we propose the Context-driven Sparse Decoding Network (CSDNet) for accurate grounding through multimodal context-aware feature extraction and text-guided sparse reasoning. To alleviate target feature confusion, a Text-aware Fusion Module (TFM) is introduced to refine the visual features using textual cues related to the image context. In addition, a Context-enhanced Interaction Module (CIM) is proposed to harmonize the differences between remote sensing images and text by modeling multimodal contexts. To tackle surface information redundancy, a Text-guided Sparse Decoder (TSD) is developed, which decouples image resolution from reasoning complexity by performing sparse sampling under text guidance. Extensive experiments on DIOR-RSVG, OPT-RSVG, and VRSBench benchmarks demonstrate the effectiveness of CSDNet. Remarkably, CSDNet utilizes only 5.12% of the visual features in performing cross-modal reasoning about the target object. The code is available at https://github.com/WUTCM-Lab/CSDNet.