SGN-CIRL: Scene Graph-Based Navigation with Curriculum, Imitation, and Reinforcement Learning

Abstract

The 3D scene graph models spatial relationships between objects, enabling the agent to efficiently navigate in a partially observable environment and predict the location of the target object. This paper proposes an original framework named SGN-CIRL (Scene Graph-based Navigation with Curriculum, Imitation, and Reinforcement Learning) for mapless reinforcement-learning-based robot navigation with a learnable representation of an open-vocabulary 3D scene graph. To accelerate and stabilize the training of reinforcement learning-based algorithms, the framework also employs imitation learning and curriculum learning. Imitation learning enables the agent to learn from demonstrations, while curriculum learning structures the training process by gradually increasing task complexity from simple to more advanced scenarios. Numerical experiments conducted in the Isaac Sim environment show that using a 3D scene graph for reinforcement learning significantly increases the success rate in difficult navigation cases.