Reinforcement learning-based optimal formation control of multiple robotic rollers in cooperative rolling compaction

Abstract

For the sake of enhancing the rolling compaction quality and operation efficiency in infrastructure construction, this paper addresses the issue of optimal formation control for cooperative rolling compaction of a group of robotic rollers (RRs) by a combination of the reinforcement learning (RL)-based tracking control technique and the virtual structure method. The RR’s kinematic model is first established by fully considering the structural characteristics of the active revolute joint. Via the kinematic model and the virtual structure method, formation control of multiple RRs is formulated as path-following control with respect to their corresponding node in the desired rolling compaction formation shape. Then, optimal formation control policies of RRs are derived by the value functional that is the solution to tracking Hamilton–Jacobi-Bellman equation. By resorting to the RL-based tracking control, approximate optimal control policies are obtained by forward-in-time online neural network estimation of the value functional. Locally uniform ultimate boundedness of the closed-loop formation error system is analyzed rigorously by the Lyapunov technique. Finally, numerical simulation results are presented for three-RR cooperative rolling compaction of a clay core wall dam in Qianping reservoir to show the effectiveness of the main results of this paper.