A Safety-Enhanced Reinforcement Learning Framework for Vision-Language Models in Autonomous Driving

Abstract

While vision-language models (VLMs) demonstrate remarkable capabilities in autonomous driving applications, they face significant challenges in safety-critical environments due to their insufficient risk awareness, susceptibility to visual distractions, and challenges in effectively balancing safety requirements with learning efficiency. To mitigate these challenges, we propose safe-GRPO, a safe-aware group relative policy optimization framework specifically designed for safe decision-making in VLM-based driving systems. Safe-GRPO introduces three key innovations: i) an uncertainty-aware, safety-sensitive reward mechanism that dynamically adjusts reward and penalty scales according to both risk severity and model uncertainty; ii) an uncertainty-guided noise-enhanced training method that adaptively improves robustness against visual interferences; and iii) a risk-aware dynamic clipping strategy that controls policy updates based on scenario-specific risks. Extensive experiments on an enhanced real-world scenarios with driving theory tests dataset (3926 risk-annotated scenarios) show that safe-GRPO significantly outperforms existing approaches in decision accuracy, robustness, and risk prediction accuracy-achieving a 75.14% safe decision rate in high-risk scenarios, demonstrating notable advancements towards reliable real-world autonomous driving applications.