Q-Learning Based Adaptive Kalman Filtering With Adaptive Window Length

In this article, we propose an adaptive Kalman filtering with adaptive window length based on Q-learning for dynamic systems with unknown model information. The iteration step length of the Q-function is quantitatively adjusted through the influence function. The adaptive Kalman filtering algorithm is used to set an appropriate weight matrix for the Q-function to estimate unknown model parameters. One numerical example and a practice-oriented case are given to illustrate the effectiveness of the proposed method. It is shown that this filtering can provide state estimates of best accuracy among all the compared methods when the model mismatch and noise statistical characteristics change.