Bridging the Gap Between Semantics and Geometry in SLAM: A Semantic-Geometric Tight-Coupling Monocular Visual Object SLAM System

Abstract

Existing object-level simultaneous localization and mapping (SLAM) methods often overlook the correspondence between semantic information and geometric features, resulting in a significant gap between them within SLAM frameworks. To tackle this issue, this article proposes, a semantic-geometric tight-coupling monocular visual object SLAM system, (TiMoSLAM), which considers a rigorous correspondence between semantics and geometry across all steps of SLAM. Initially, a general semantic relation graph (SRG) is developed to consistently represent semantic information alongside geometric features. Detailed analyzes on complete constraints of the geometric feature combinations on estimation of 3-D cuboid model are performed. Subsequently, a compound hypothesis tree is proposed to incrementally construct the object-specific SRG and concurrently estimate the 3-D cuboid model of an object, ensuing semantic-geometric consistency in object representation and estimation. Special attention is given to the matching errors between geometric features and objects during the optimization of camera poses and object parameters. The effectiveness of this method is validated on various datasets, as well as in real-world environments.