Aerial Vision-Language Navigation With a Unified Framework for Spatial, Temporal and Embodied Reasoning

Abstract

Aerial Vision-and-Language Navigation (VLN) aims to enable uncrewed aerial vehicles (UAVs) to interpret natural language instructions and navigate complex urban environments using onboard visual observations. This task holds promise for real-world applications such as low-altitude inspection, search-and-rescue, and autonomous aerial delivery. Existing methods often rely on panoramic images, depth information, or odometry to support spatial reasoning and action planning. Such reliance on additional sensing modalities increases system cost and integration complexity. In this work, we present a unified aerial VLN framework that operates solely on egocentric monocular RGB observations and natural language instructions. The model formulates navigation as a next-token prediction problem, jointly optimizing spatial perception, trajectory reasoning, and action prediction through prompt-guided multi-task learning. Moreover, we propose a keyframe selection strategy to reduce visual redundancy by retaining semantically informative frames, along with an action merging and label reweighting mechanism that mitigates long-tailed supervision imbalance and facilitates stable multi-task co-training. Extensive experiments on the AerialVLN and OpenFly benchmarks validate the effectiveness of our method. Under the challenging monocular RGB-only setting, our model achieves strong results across both seen and unseen environments. It significantly outperforms existing monocular baselines and narrows the performance gap with state-of-the-art panoramic RGB-D counterparts. Comprehensive ablation studies further validate the contribution of our architectural design and choices. Our code is publicly available at https://github.com/return-sleep/AeroAct