Predicting the long-term robustness of visual features

Abstract

Many vision based localization methods extract local visual features to build a sparse map of the environment and estimate the position of the camera from feature correspondences. However, the majority of features is typically only detectable for short time-frames so that most information in the map becomes obsolete over longer periods of time. Long-term localization is therefore a challenging problem especially in outdoor scenarios where the appearance of the environment can change drastically due to different day times, weather conditions or seasonal effects. We propose to learn a model of stable and unstable feature characteristics from texture and color information around detected interest points that allows to predict the robustness of visual features. The model can be incorporated into the conventional feature extraction and matching process to reject potentially unstable features during the mapping phase. The application of the additional filtering step yields more compact maps and therefore reduces the probability of false positive matches, which can cause complete failure of a localization system. The model is trained with recordings of a train journey on the same track across seasons which facilitates the identification of stable and unstable features. Experiments on data of the same domain demonstrate the generalization capabilities of the learned characteristics.