Language Integration in Remote Sensing: Tasks, datasets, and future directions

Abstract

The emerging field of vision–language models, which combines computer vision and natural language processing (NLP), has gained significant interest and exploration. This integration has opened up new research opportunities, particularly in remote sensing (RS), where it has the potential to enhance RS systems’ capabilities. In this context, this article presents a comprehensive review of more than 100 articles focusing on the integration of NLP techniques into RS understanding research. The review covers various vision–language modeling tasks, including but not limited to RS image captioning, RS text-to-image retrieval, RS visual question answering (VQA), and RS image generation. For each task, the review provides a summary of the state-of-the-art developments, including methods, evaluation metrics, datasets, and experimental results on benchmark datasets. The review is concluded by discussing the key challenges and highlighting potential research directions for future development, with the aim of inspiring further research in this important field.