Cross-domain UAV pose estimation: A novel attempt in UAV visual localization

Abstract

With the rapid advancement of depth estimation algorithms and continuous improvements in devices such as LiDAR and depth cameras, the acquisition of high-quality 3D models has become increasingly accessible. This progress opens up new opportunities for leveraging cross-domain matching between images and point clouds to estimate the pose of Unmanned Aerial Vehicles (UAVs) for visual localization. In this context, we propose a novel cross-domain descriptor that facilitates the fusion and matching of features across modalities. Building upon this approach, we designed a dual-branch UAV localization pipeline that incorporates an object detection strategy to extract more reliable feature points from the scene. Additionally, we constructed two new datasets specifically tailored for UAV-based aerial applications. The first dataset is manually annotated and focuses on training and evaluating object detection models from an aerial perspective, while the second dataset contains approximately 1.7 million 2D-3D correspondences from diverse scenarios, offering a rich collection of training and evaluation samples. Extensive experiments on public UAV datasets demonstrate that, compared to existing descriptors, our method not only achieves superior pose estimation accuracy through a coarse-to-fine image matching strategy but also enables robust pose estimation by directly matching images and point clouds to obtain accurate 2D-3D correspondences. Moreover, the incorporation of object detection strategies significantly enhances pose estimation accuracy and demonstrates increased resilience to interference in complex environments. Our datasets and code will be publicly available at https://github.com/lwhhhh13/Cross-Domain-UAV-Pose-Estimation.