Fixed-time learning for safe time-critical verification using reachability analysis

Abstract

In this paper, we address a safe time-critical control problem using reachability analysis and design a reinforcement learning-based mechanism for learning online and in fixed-time the solution to the safe time-critical control problem. Safety is ensured by determining a set of states for which there exists an admissible control law generating a system trajectory that does not reach a set of forbidden states at a user-prescribed time instant. Specifically, we cast our safe time-critical problem as a Mayer optimal feedback control problem whose solution satisfies the Hamilton–Jacobi–Bellman (HJB) equation and characterizes the set of safe states. Since the HJB equation is generally difficult to solve, we develop an online critic-only reinforcement learning-based algorithm for simultaneously learning the solution to the HJB equation and the safe set in a fixed time. In particular, we introduce a non-Lipschitz experience replay-based learning law utilizing recorded and current data for updating the critic weights to learn the value function and the safe set. The non-Lipschitz property of the dynamics gives rise to fixed-time convergence, whereas the experience replay-based approach eliminates the need to satisfy the persistence of excitation condition provided that a recorded data set is sufficiently rich. Simulation results illustrate the efficacy of the proposed approach to the problem of fixed-wing unmanned aerial vehicle collision avoidance.