Safe reinforcement learning for discrete-time nonlinear zero-sum games with unknown state constraints and asymmetric input constraints

Abstract

In this paper, we propose a novel safe reinforcement learning (RL) algorithm for discrete-time nonlinear zero-sum games with unknown state constraints and asymmetric input constraints. To address this constrained optimal problem, we adopt a value iteration framework based on neural networks, incorporating a critic-only structure. Given the unknown safety constraints, we tackle the state constraint issue by introducing a neural network-based control barrier function (CBF) using collected data to augment the reward function. Furthermore, by leveraging the non-monotonic increasing property of the value function, we ensure the system’s safety. Additionally, we construct a non-quadratic function to further augment the reward function, thereby satisfying the asymmetric input constraints. This paper also includes a series of theoretical proofs that rigorously demonstrate the convergence and safety of the proposed algorithm. Finally, experiments conducted under different scenarios and parameter settings, compared with existing algorithms, validate the algorithm’s effectiveness and safety.