Reinforcement learning based attitude fault-tolerant control of spacecraft with unknown system model

With the increasing reliability and safety requirements of spacecraft control system, it is urgent to study effective fault-tolerant control methods to ensure that the control system can still maintain high control performance when the actuator fails. Considering the uncertainty and suddenness of faults, it is very important for fault-tolerant control to have strong adaptability and high real-time performance. Therefore, a fault-tolerant attitude controller based on reinforcement learning for a spacecraft with unknown system model is proposed in this paper. Firstly, by ignoring the effects of inertia uncertainty, actuator faults, and external disturbances, the reinforcement learning algorithm is combined with optimal control to design an offline approximate optimal control policy for the known nominal system. Next, a neural network-based observer is designed to estimate the system input matrix and approximate the unknown system dynamics, eliminating the offline nominal controller's dependency on the system model and mitigating the impact of multiplicative actuator faults on attitude control. Subsequently, the offline nominal controller is employed as the initial control strategy, and the parameters of the critic network are updated online using the RLS method. This approach results in an online reinforcement learning controller that enhances the algorithm's real-time performance. In addition, by utilizing the partial disturbances estimated by the neural network-based observer, a feedforward compensation algorithm is incorporated to counteract the adverse effects of additive actuator faults and external disturbance torques on attitude control performance, completing the online fault-tolerant control scheme based on reinforcement learning. Lastly, the stability of the control system is proved by the Lyapunov method, and the validity of the proposed fault-tolerant control is illustrated through simulations.