Dynamic Historical Data-Based Reinforcement Learning for Pursuit–Evasion Games of Nonholonomic Vehicles With Input Saturation

Abstract

This article studies the pursuit–evasion game involving nonholonomic vehicles constrained by input saturation, aiming for the pursuer to intercept an evasive opponent. Unlike the previous game research neglecting the practical kinematic constraints, a coupled nonlinear system is formulated to elucidate the interaction dynamics between the players. After that, the optimal control strategies are derived by solving the Hamilton–Jacobi–Isaacs (HJI) equation linked to a special nonquadratic cost function. The Nash equilibrium analysis and finite-time capturability are conducted. To learn the optimal pursuit–evasion strategy pair, a fixed-time convergent reinforcement learning (RL) algorithm is proposed, which leverages a novel residual design to facilitate weight updates by collecting and evaluating current and historical data based on information quality. Compared with the existing RL methods that suffer from sluggish convergence due to an asymptotic learning rule and the stringent persistent excitation (PE) condition, the proposed RL relaxes the PE to an easily achievable and online verifiable finite excitation (FE) condition, allowing rapid weight convergence within a fixed period. Simulations and comparisons validate the effectiveness and superiority of the proposed method, showing a 61% reduction in convergence time in contrast to the prevailing RL schemes.