MCNN-VIO: A High-Accuracy Multi-Camera Visual-Inertial Odometry With Neural Networks

Abstract

Visual-inertial odometry (VIO) has the advantages of small size and low hardware cost as one of the methods for state estimation. Especially, the accuracy of the filtering-based system is high and its computational load is low. The multiple cameras configured in different directions are always applied to expand the observation scope so that the poor robustness of single-camera estimation and the abilities of object perception will be improved. However, multiple cameras will track more features, which not only increases calculation but also adds many features with large errors into programs, leading the low accuracy. In this article, a new framework, multi-camera with neural networks VIO (MCNN-VIO), based on multistate constraint Kalman filter (MSCKF) is proposed. It fuses two stereo cameras with non-overlapping fields of view (FoV) and an inertial measurement unit (IMU). The feature processing strategies between different cameras have been redesigned. The method is capable of using neural networks to intelligently select features in a variety of complex environments. Besides, a novel strategy of feature selection is proposed to obtain the closest poses to the true value for network training. This strategy can find the optimal solution in a limited number of stochastic and inclined combinations. The method was tested in scenes with both rich features and challenging darkness. The experimental results show that the method exhibits higher accuracy and better robustness compared to the multi-camera configuration of the conventional algorithm. Meanwhile, it maintains a competitive performance and low calculation cost compared to a single-camera version.