Improved DDPG-Based Path Planning for Mobile Robots

Abstract

With the rapid advancement of robotics technology, path planning has attracted extensive research attention. Reinforcement learning, owing to its ability to acquire optimal policies through continuous interaction with the environment, offers a promising solution for path planning in environments with incomplete or unknown information. However, reinforcement learning-based path planning methods often suffer from high training complexity and low utilization of effective samples. To address these issues, this paper proposes an improved deep reinforcement learning (DRL) algorithm. The proposed approach builds upon the deep deterministic policy gradient (DDPG) algorithm and incorporates a short-term goal planning strategy based on local perceptual information, which decomposes the global navigation task into multiple short-term subgoals, thereby reducing task complexity and enhancing learning efficiency. Furthermore, a reward function integrating the artificial potential field (APF) method is designed to improve obstacle avoidance capability. To tackle the low utilization of effective experiences in DDPG, a dual experience pool strategy is introduced to improve experience utilization efficiency and accelerate model training. The parameters for short-term goal selection are optimized through multiple comparative experiments, and the proposed method is evaluated against several DRL-based path planning approaches in a static environment. Experimental results demonstrate that the improved algorithm significantly accelerates convergence. Moreover, dynamic environment simulation experiments verify that the proposed algorithm can effectively avoid moving obstacles and achieve safe navigation to the target position.