Hybrid sparse monocular visual odometry with online photometric calibration

Most monocular visual Simultaneous Localization and Mapping (vSLAM) and visual odometry (VO) algorithms focus on either feature-based methods or direct methods. Hybrid (semi-direct) approach is less studied although it is equally important. In this paper, a hybrid sparse visual odometry (HSO) algorithm with online photometric calibration is proposed for monocular vision. HSO introduces two novel measures, that is, direct image alignment with adaptive mode selection and image photometric description using ratio factors, to enhance the robustness against dramatic image intensity changes and motion blur. Moreover, HSO is able to establish pose constraints between keyframes far apart in time and space by using KLT tracking enhanced with a local-global brightness consistency. The convergence speed of candidate map points is adopted as the basis for keyframe selection, which strengthens the coordination between the front end and the back end. Photometric calibration is elegantly integrated into the VO system working in tandem: (1) Photometric interference from the camera, such as vignetting and changes in exposure time, is accurately calibrated and compensated in HSO, thereby improving the accuracy and robustness of VO. (2) On the other hand, VO provides pre-calculated data for the photometric calibration algorithm, which reduces resource consumption and improves the estimation accuracy of photometric parameters. Extensive experiments are performed on various public datasets to evaluate the proposed HSO against the state-of-the-art monocular vSLAM/VO and online photometric calibration methods. The results show that the proposed HSO achieves superior performance on VO and photometric calibration in terms of accuracy, robustness, and efficiency, being comparable with the state-of-the-art VO/vSLAM systems. We open source HSO for the benefit of the community.