Lyapunov-based adaptive deep system identification for approximate dynamic programming

Recent developments in approximate dynamic programming (ADP) use deep neural network (DNN)-based system identifiers to solve the infinite horizon state regulation problem; however, the DNN weights do not continually adjust for all layers. In this paper, ADP is performed using a Lyapunov-based DNN (Lb-DNN) adaptive identifier that involves online weight updates. Provided the Jacobian of the Lb-DNN satisfies the persistence of excitation condition, the Lb-DNN weights exponentially converge to a residual approximation error, and the corresponding control policy converges to a neighborhood of the optimal policy. Simulation results show that the Lb-DNN yields 49.85% improved root mean squared (RMS) function approximation error in comparison to a baseline ADP DNN result and faster convergence of the RMS regulation error, RMS controller error, and RMS function approximation error.