Deep Reinforcement Learning for Localisability-Aware Mapless Navigation

Abstract

Mapless navigation refers to the task of searching for a collision free path without relying on a pre-defined map. Most current works of mapless navigation assume accurate ground-truth localisation is available. However, this is not true, especially for indoor environments, where simultaneous localisation and mapping (SLAM) is needed for location estimation, which highly relies on the richness of environment features. In this work, we propose a novel deep reinforcement learning (DRL) based mapless navigation method without relying on the assumption of the availability of localisation. Our method utilises RGB-D based Oriented FAST and Rotated BRIEF (ORB) SLAM2 for robot localisation. Our policy effectively guides the robot's movement towards the target while enhancing robot pose estimation by considering the quality of the observed features along the selected paths. To facilitate policy training, we propose a compact state representation based on the spatial distributions of map points, which enhances the robot's awareness of areas with reliable map points. Furthermore, we suggest incorporating the relative pose error into the reward function. In this way, the policy will be more responsive to each single action. In addition, rather than utilising a pre-set threshold, we adopt a dynamic threshold to improve the policy's adaptability to variations in SLAM performance across different environments. The experiments in localisation challenging environments have demonstrated the remarkable performance of our proposed method. It outperforms the related DRL based methods in terms of success rate.