Optimization control with multi-constraint of aeroengine acceleration process based on reinforcement learning

Abstract

With the development of Reinforcement Learning (RL), it becomes able to solve the continuous action space problem and shows strong ability in dealing with complex nonlinear control problem. Based on the Deep Deterministic Policy Gradient (DDPG) algorithm, a novel scheme of aeroengine acceleration controller is proposed in this paper. According to the characteristics of the engine acceleration stage, the reward function is constructed, and the state parameters are updated in the form of sliding window to reduce the sensitivity of the network to noise. DDPG adopts actor-critic framework, critic calculates value function by the deep neural network, actor outputs action command and forms a closed-loop control system with the engine. The method is verified by digital simulation at ground condition and the results demonstrate that compared with the traditional PID controller, the acceleration time of DDPG controller is reduced by 41.56%. Additionally, the network converges within 400 steps.