Disturbance Compensation-Based Deep Reinforcement Learning Control Strategy for Underactuated Overhead Crane Systems: Design and Experiments

Abstract

Transportation systems often exhibit uncertainty, dynamics, and nonlinearity, making them highly susceptible to perturbations. Overhead crane systems, being a crucial mode of transportation, find wide application in engineering practice. This paper proposes a compound control strategy for underactuated overhead crane systems, combining a deep reinforcement learning controller with a novel observer-based disturbance compensation algorithm. The aim is to suppress model uncertainties and external disturbances. Initially, an equivalent state variable is constructed to stabilize the overhead crane system, simplifying controller design. Subsequently, the model uncertainties and external disturbances are treated as generalized disturbances and incorporated as a new state. This state is estimated and compensated through a super-twisting extended state observer (STESO) with finite-time convergence. Lyapunov-based analysis demonstrates that the established state observer can reduce the residual estimate error to zero. Moreover, the Deep Deterministic Policy Gradient (DDPG) algorithm, a form of deep reinforcement learning, is employed to effectively eliminate tracking errors. The proposed scheme enables the underactuated system to achieve satisfactory performance, with adaptive control effort suited to different control states and disturbances. Simulation and hardware experiments are conducted, comparing the proposed control strategy with traditional methods, to illustrate its effectiveness and robustness.