Active Disturbance Rejection Control Based on Twin-Delayed Deep Deterministic Policy Gradient for an Exoskeleton

The study of exoskeletons has been a popular topic worldwide. However, there is still a long way to go before exoskeletons can be widely used. One of the major challenges is control, and there is no specific research trend for controlling exoskeletons. In this paper, we propose a novel exoskeleton control strategy that combines Active Disturbance Rejection Control (ADRC) and Deep Reinforcement Learning (DRL). The dynamic model of the exoskeleton is constructed, followed with the design of the ADRC. To automatically adjust the control parameters of the ADRC, the Twin-Delayed Deep Deterministic Policy Gradient (TD3) is utilized. Then a reward function is defined in terms of the joint angle, angular velocity, and their errors to the desired values, to maximize the accuracy of the joint angle. In the simulations and experiments, a conventional ADRC, and ADRC based on Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) were carried out for comparison with the proposed control method. The results of the tests show that TD3-ADRC has a rapid response, small overshoot, and low Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) followed with the desired, demonstrating the superiority of the proposed control method for the self-learning control of exoskeleton.