HAD-Gen: Human-like and diverse driving behavior modeling for controllable scenario generation

Simulation-based testing has emerged as an essential tool for verifying and validating autonomous vehicles (AVs). However, contemporary methodologies, such as deterministic and imitation learning-based driver models, struggle to capture the variability of human-like driving behavior. Given these challenges, we propose HAD-Gen, a general framework for realistic traffic scenario generation that simulates diverse human-like driving behaviors. The framework first clusters the vehicle trajectory data into different driving styles according to safety features. It then employs maximum entropy inverse reinforcement learning on each of the clusters to learn the reward function corresponding to each driving style. Using these reward functions, the method integrates offline reinforcement learning pre-training and multi-agent reinforcement learning algorithms to obtain general and robust driving policies. Multi-perspective simulation results in highway scenarios show that our proposed scenario generation framework can generate diverse, human-like driving behaviors with strong generalization capability. The proposed framework achieved a 90.96% goal-reaching rate, an off-road rate of 2.08%, and a collision rate of 6.91% in new unseen driving scenarios, outperforming prior approaches by over 20% in goal-reaching performance. The source code is released at https://github.com/RoboSafe-Lab/HAD-Gen.