HeightCLIP: A vision-language network for height estimation from monocular remote sensing imagery

Abstract

Accurate height estimation from monocular remote sensing imagery is a fundamental task with broad applications in urban planning and management. However, existing vision-based methods rely solely on visual features, which limits their ability to fully exploit height-related semantic information. Vision-language models, e.g., CLIP, offer a promising alternative by leveraging multimodal information, yet directly adapting them to height regression remains challenging due to their weak perception of continuous numerical values. To overcome this limitation, we propose HeightCLIP, a novel vision-language framework tailored for height estimation from monocular remote sensing imagery. The task-adaptive textual prompts and image-side representations are optimized under height supervision to derive pixel-text similarity features for accurate height estimation. Specifically, we design an interval-based prompting strategy that integrates learnable context tokens and a fixed natural language template with parameterized interval values, allowing CLIP to better comprehend height-related semantics. In addition, we incorporate a reciprocal cross-attention fusion mechanism to facilitate bidirectional interaction between visual and textual features, thereby improving cross-modal feature integration. Finally, building upon this enriched feature space, we impose cross-modal consistency constraints to ensure pixel-level correspondence between height descriptions and visual features, providing more informative representations for subsequent height prediction. Extensive experiments on three benchmark datasets i.e., Vaihingen, Potsdam, and Data Fusion Contest 2019 (DFC2019), demonstrate that HeightCLIP achieves competitive or superior performance compared with existing state-of-the-art methods, and comprehensive ablation studies confirm the effectiveness of each proposed component. The source code will be made available at https://github.com/songtaowhu/HeightCLIP.