Goal-driven navigation via variational sparse Q network and transfer learning

Abstract

Compared to traditional map-based, goal-driven navigation methods, deep reinforcement learning (DRL)-based goal-driven navigation for mobile robots offers the advantage of not relying on prior map information, it enables autonomous decision-making through continuous interaction with the environment. However, DRL-based goal-driven navigation faces significant challenges in terms of low generalization ability and learning inefficiency. In this paper, we propose a DRL approach, variational sparsity Q network (VSQN), which leverages variational inference and transfer learning to achieve efficient goal-driven navigation. The variational inference framework models weight uncertainty within the network, thereby enhancing the agent’s generalization capability. Furthermore, a hierarchical learning network framework is adopted, and transfer learning is employed to incorporate prior knowledge from a pre-trained model into new navigation tasks. This enables the agent to rapidly adapt to novel tasks without the need for fine-tuning after selecting an optimal sub-goal. This improves the agent’s initial performance in previously unseen navigation tasks. The experimental results indicate that the proposed method achieves a success rate (SR) of 76% and a success weighted by inverse path length (SPL) of 0.52 in previously unencountered environments and target locations within the grid environment, and an SR of 81% with an SPL of 0.30 in the AI2-THOR environment. These findings demonstrate that the method substantially enhances the agent’s generalization capability.