Generation of naturalistic and critical boundary scenarios: A bi-level adaptive deep reinforcement learning method

Abstract

The complexities of real-world driving environments, coupled with a limited availability of naturalistic and critical test scenarios, have long hindered unbiased and effective comprehensive performance evaluations. In this work, we propose a bi-level adaptive deep reinforcement learning (BADRL) framework designed to generate realistic and diverse critical boundary scenarios. The method involves training AI-driven background agents to impartially assess the overall performance of autonomous vehicles. By leveraging naturalistic driving data, these agents acquire realistic driving behaviors via a neural model that encapsulates naturalistic driving patterns. To enrich the authenticity and diversity of the test scenarios, a wide array of traffic participants, encompassing vehicles, pedestrians, and bicycles, are meticulously modeled and portrayed to engage in intricate interactive behaviors with the tested autonomous vehicles. To address the challenges of high-dimensional environments, we introduce a scenario complexity model that assesses relative complexity in real time. This model enables the upper-level neural network in BADRL to dynamically escalate scenario complexity, with the resulting scenarios subsequently processed by lower-level models to optimize the actions of primary traffic participants. The BADRL method enables online real-time generation of naturalistic and critical boundary scenarios. Extensive simulation experiments validate the effectiveness of the BADRL approach in diverse driving environments, with results indicating an improvement in the efficiency of critical boundary scenario generation by approximately 15.89 % compared to state-of-the-art methods.