City-level aerial geo-localization based on map matching network

Abstract

Autonomous localization of aircraft relies on precise geo-localization, and under Global Navigation Satellite System (GNSS)-denied conditions, visual localization methods are among the most important techniques for aircraft autonomous localization. Global visual localization typically relies on pre-established 3D maps, which require significant storage and computational overhead, limiting the applicability of aerial visual localization. Therefore, we propose a visual localization method based on OpenStreetMap, an openly accessible 2D map. This method not only enables localization in the absence of GNSS but also has lower storage and computational requirements compared to 3D map-based visual methods. This makes our approach feasible for visual geo-localization at the urban scale. We designed a neural network model based on the Vision Transformer (ViT) to extract features from aerial images and 2D maps for fast matching and retrieval, thereby estimating the global geo-location of the aerial images. Additionally, we employ particle filtering to fuse location estimates from map retrieval, visual odometry, and GNSS, achieving higher-precision real-time localization. Moreover, we collected aerial images and map tiles covering over 1000 square kilometers from the urban and suburban areas of four large cities, creating a novel aerial image-to-map matching dataset. Experiments show that, compared to the current state-of-the-art methods, our map retrieval network achieves a higher average recall rate on the dataset. In GNSS-denied conditions, our multi-source fusion localization method can achieve real-time global geo-localization at the urban scale, and under weak GNSS signals, our method provides significantly higher localization accuracy than GNSS alone.