Modeling framework of human driving behavior based on Deep Maximum Entropy Inverse Reinforcement Learning

Driving behavior modeling is extremely crucial for designing safe, intelligent, and personalized autonomous driving systems. In this paper, a modeling framework based on Markov Decision Processes (MDPs) is introduced that emulates drivers’ decision-making processes. The framework combines the Deep Maximum Entropy Inverse Reinforcement Learning (Deep MEIRL) and a reinforcement learning algorithm-proximal strategy optimization (PPO). A neural network structure is customized for Deep MEIRL, which uses the velocity of the ego vehicle, the pedestrian position, the velocity of surrounding vehicles, the lateral distance, the surrounding vehicles’ type, and the distance to the crosswalk to recover the nonlinear reward function. The dataset of drone-based video footage is collected in Xi’an (China) to train and validate the framework. The outcomes demonstrate that Deep MEIRL-PPO outperforms traditional modeling frameworks (Maximum Entropy Inverse Reinforcement Learning (MEIRL) - PPO) in modeling and predicting human driving behavior. Specifically, in predicting human driving behavior, Deep MEIRL-PPO outperforms MEIRL-PPO by 50.71% and 43.90% on the basis of the MAE and HD, respectively. Furthermore, it is discovered that Deep MEIRL-PPO accurately learns the behavior of human drivers avoiding potential conflicts when lines of sight are occluded. This research can contribute to aiding self-driving vehicles in learning human driving behavior and avoiding unforeseen risks.