Online Actor-critic Network Algorithm to Solve infinite-Horizon Optimal Tracking Control Problem for Discrete-time Systems

Abstract

A novel value function is cleverly defined in this paper to eliminate the tracking error for the infinite-Horizon optimal tracking control problem by the adaptive dynamic programming (ADP) technique. First, instead of using the original quadratic form, as in the existing ADP approaches, a novel formulation of the cost function is obtained by error accumulation. Second, the performance index and control input are approached by the critic-actor neural network (NN). Finally, the system state and network weight errors are uniformly ultimately bounded (UUB), according to theoretical studies. This is the first version of the stability analysis of the ADP method by constructing novel value function to eliminate tracking error. The theoretical arguments are also supported by a simulation outcome.