Deep reinforcement learning-based trajectory planning for double pendulum cranes: Design and experiments

Abstract

Since cranes usually possess double pendulum dynamics, the mass of the payload often changes and there are frequently lifting/lowering operations simultaneously. Moreover, most crane systems are driven by motors, whose velocities and accelerations are often limited. To solve the above problems, this paper proposes a deep reinforcement learning (DRL) reference trajectory generation method based on virtual–physical joint training. Firstly, a DRL module based on deep deterministic policy gradient (DDPG), along with a double pendulum crane dynamic model and an adaptive controller, are established within the virtual environment for training the reference trajectory. When the reward reaches the threshold, the training is switched to the physical environment to further optimize the reference trajectory, realizing the swing suppression of the hook and the payload. In addition, to satisfy the performance of the drive motors, velocity and acceleration thresholds are set to constrain the performance of the drive motors. Finally, in view of the fact that the operation process of the double pendulum crane is divided into payload transportation and payload loading/unloading, an event-triggering mechanism is designed to switch different control policies in accordance with different operation processes, thus reducing the consumption of computing resources. Through experiments on the actual double pendulum crane and comparison with the existing reference trajectories and input shapers, the superiority of this method is demonstrated.