An optimal guidance law based on deep reinforcement learning for compensating the lag in time-varying slope path following

Abstract

During the carrier landing process, the carrier motion induces real-time variations in the desired slope path. Considering the inherent lag in aircraft position adjustments, this paper proposes an optimal guidance law based on deep reinforcement learning (DRL) to compensate for the lag. First, the carrier landing process is modeled as a Finite Markov Decision Process (FMDP), and a comprehensive DRL framework is developed. Second, a novel Soft Actor-Critic (LA-SAC) method enhanced with the Long Short-Term Memory (LSTM) network and the attention mechanism (AM) is introduced. The method extracts the deck motion features with the LSTM network and adjusts the weights of different state data with AM to improve learning efficiency. Additionally, a distributed neural network is designed to integrate deck motion prediction and compensation, avoiding the complexity of parameter tuning in conventional methods. LA-SAC leverages full-dimensional data to train the network and derive an optimal guidance law. Finally, the superiority of the proposed method has been verified in a semi-physical simulation platform. Compared to DRL baselines, LA-SAC achieves faster convergence and derives a superior guidance policy. Compared to conventional methods, the proposed method provides a more significant lead margin to reduce landing errors. Furthermore, the ablation experiments confirmed the effectiveness of the LSTM network and AM modules, and the real-time analysis validated the practicality of the LA-SAC algorithm in actual implementation.