Keyframes retrieval for robust long-term visual localization in changing conditions

Abstract

Appearance changes are a challenge for visual localization in outdoor environments. Revisiting familiar places but retrieving keyframes that were taken under different environmental condition can result in inaccurate localization. To overcome this difficulty, we propose a localization approach able to take advantage of a visual landmark map composed of $N$ sequences gathered at different times and conditions. During this localization process, we exploit information collected in the beginning of the trajectory to compute a ranking function which will be used in the rest of the trajectory to retrieve from the map the keyframes that maximise the number of matched points. The retrieval depends on the geometric distance between the pose of the keyframe and the current pose of the vehicle, and the similarity of this keyframe with the current environmental condition. The results demonstrate that our approach has significantly improved localization performance in challenging conditions (snow, rain, change of season …).