Neural Network–Based Reinforcement Iterative Learning Fault Estimation Scheme for Nonlinear Uncertain Manipulator Systems With Time-Delay

This article investigates the problem of fault estimation in nonlinear uncertain manipulator systems with time-delay. A novel fault estimation scheme is proposed, which optimizes the iterative learning (IL) estimator performance using a neural network (NN)-based reinforcement learning (RL) approach. Specifically, first, with the purpose of enhancing robustness and adaptability of RL, a new adaptive exponential reward function is designed. Then, to improve the performance of fault estimation, speed and accuracy are designed as optimization objectives. Simultaneously, by leveraging the IL estimator, the NN is continuously optimized in each iteration process to mitigate the issues of gradient vanishing and explosion. Further, by incorporating the H$\mathrm{\infty }$ performance index into the observer, an asymptotically convergent estimated error can be attained. Finally, numerical simulations are conducted to demonstrate the effectiveness of our method.