Barrier Lyapunov Function-Based Safe Reinforcement Learning Algorithm for Autonomous Vehicles with System Uncertainty

Abstract

Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. For such safety-critical systems, it will certainly be a requirement that safe performance should be ensured even during the reinforcement learning period in the presence of system uncertainty. To address this issue, a Barrier Lyapunov Function-based safe reinforcement learning algorithm (BLF-SRL) is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges the Barrier Lyapunov Function item into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning when unknown bounded system uncertainty exists. More specifically, the overall system control is optimized with the optimized backstepping technique under the framework of Actor-Critic, which optimizes the virtual control in every backstepping subsystem. Wherein, the optimal virtual control is decomposed into Barrier Lyapunov Function items; and also with an adaptive item to be learned with deep neural networks, which achieves safe exploration during the learning process. Eventually, the principle of Bellman optimality is satisfied through iteratively updating the independently approximated actor and critic to solve the Hamilton-Jacobi-Bellman equation in adaptive dynamic programming. More notably, the variance of control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with motion control problems for autonomous vehicles through appropriate comparison simulations.