Self-Supervised Visual Odometry Based on Scene Appearance-Structure Incremental Fusion

Abstract

Self-supervised visual odometry (VO) has exhibited remarkable benefits over supervised methods, surpassing the reliance on the annotated ground-truth of training data. However, most existing self-supervised VO methods, namely scene appearance-based methods, have limitations in exploiting the complementary properties of cross-modal information between scene appearance and structure. To this end, we propose a novel self-supervised VO based on scene appearance-structure incremental fusion scheme. Specifically, a Global-Local Context awareness-based Depth estimation Network (GLC-DN) is designed to introduce the scene structural cues, thus laying the foundation for realizing the scene appearance-structure incremental fusion. Then, a Dual stream Pose estimation Network based on Scene Appearance-Structure Incremental Fusion (SASIF-DPN) is devised, which consists of a Dual Stream Network (DSN) and multiple Cross-Modal Complementary Fusion Modules (CM-CFMs). CM-CFM fully leverages the complementary properties between the RGB information and the predicted depth information, and the combination of multiple CM-CFMs facilitates the information interaction between the two modalities in an incremental fusion manner. Detailed evaluations of GLC-DN and SASIF-DPN provably confirm the effectiveness and design principles of each component we propose. Extensive comparison experiments have also been conducted, which clearly verify the superiority of our method compared to current counterparts.