Morphing aircraft acceleration and deceleration task morphing strategy using a reinforcement learning method

Abstract

This paper proposes a design scheme for a whole morphing strategy based on the reinforcement learning (RL) method. A novel morphing aircraft is designed, and its nonlinear dynamic equations are established based on the calculated aerodynamic data. Further, a soft actor critic (SAC) approach is utilized to design the scheme, whose structure consists of the environment, the agent, and the reward function. In the environment design part, the incremental backstepping approach is employed to design the morphing aircraft controller. The safety and feasibility of deployment are verified. In the agent design part, in addition to using the entropy regularization RL algorithm, the generalization ability of the agent is enhanced in three ways: adding environmental noise, adding control command randomness, and adding output momentum terms. For the reward function, a structure with dynamic and steady-state performance is designed to accurately describe the aircraft dynamics. Finally, the designed SAC strategy is verified under the acceleration and deceleration tasks and compared with a GA and PPO strategy. Simulation results validate the effectiveness and superiority of the designed SAC scheme.