Reinforcement learning method based on sample regularization and adaptive learning rate for AGV path planning

Abstract

This paper proposes the proximal policy optimization (PPO) method based on sample regularization (SR) and adaptive learning rate (ALR) to address the issues of limited exploration ability and slow convergence speed in Autonomous Guided Vehicle (AGV) path planning using reinforcement learning algorithms in dynamic environments. Firstly, the regularization term based on empirical samples is designed to solve the bias and imbalance issues of training samples, and the sample regularization is added to the objective function to improve the policy selectivity of the PPO algorithm, thereby increasing the AGV’s exploration ability during the training process in the working environment. Secondly, the Fisher information matrix of the Kullback-Leibler (KL) divergence approximation and the KL divergence constraint term are exploited to design the policy update mechanism based on the dynamically adjustable adaptive learning rate throughout training. The method considers the geometric structure of the parameter space and the change of the policy gradient, aiming to optimize parameter update direction and enhance convergence speed and stability of the algorithm. Finally, the AGV path planning scheme based on reinforcement learning is established for simulation verification and comparations in two-dimensional raster map and Gazebo 3D simulation environment. Simulation results verify the feasibility and superiority of the proposed method applied to the AGV path planning problem.