Critical scenarios adversarial generation method for intelligent vehicles testing based on hierarchical reinforcement architecture

Abstract

The widespread deployment of intelligent vehicles necessitates comprehensive testing across diverse driving scenarios. A significant challenge is generating critical testing scenarios to accurately evaluate vehicle performance. To overcome the limitations of existing methods, including inadequate diversity and validity, this study proposes an adversarial generation method grounded on a hierarchical reinforcement learning framework. This approach comprises three modules: a hierarchical scheduling module, a conflict prediction module, and a scenario evaluation module. The hierarchical scheduling module segments the testing procedure into guidance, adversarial, and exploration periods, effectively managing reward sparsity to promote varied scenario generation. The conflict prediction module employs kinematic conflict prediction and adaptive action strategies to enhance learning speed and efficiency, directing traffic entities in producing critical scenarios. The evaluation module assesses scenario validity and diversity by analyzing relative trajectories, temporal characteristics, and spatial configurations, in addition to employing a perception-limited model and replay testing to assess performance within the system’s operational limits. Experimental results using the HighD dataset in the highway environment demonstrate that the proposed method efficiently generates varied critical test scenarios, improving the collision rate and period contributions throughout the testing process. When producing an equivalent number of critical scenarios, the overall testing resource utilization decreases by 49.49% relative to the conventional Deep Q-Network method.