{"title":"Modeling framework of human driving behavior based on Deep Maximum Entropy Inverse Reinforcement Learning","authors":"","doi":"10.1016/j.physa.2024.130052","DOIUrl":null,"url":null,"abstract":"<div><p>Driving behavior modeling is extremely crucial for designing safe, intelligent, and personalized autonomous driving systems. In this paper, a modeling framework based on Markov Decision Processes (MDPs) is introduced that emulates drivers’ decision-making processes. The framework combines the Deep Maximum Entropy Inverse Reinforcement Learning (Deep MEIRL) and a reinforcement learning algorithm-proximal strategy optimization (PPO). A neural network structure is customized for Deep MEIRL, which uses the velocity of the ego vehicle, the pedestrian position, the velocity of surrounding vehicles, the lateral distance, the surrounding vehicles’ type, and the distance to the crosswalk to recover the nonlinear reward function. The dataset of drone-based video footage is collected in Xi’an (China) to train and validate the framework. The outcomes demonstrate that Deep MEIRL-PPO outperforms traditional modeling frameworks (Maximum Entropy Inverse Reinforcement Learning (MEIRL) - PPO) in modeling and predicting human driving behavior. Specifically, in predicting human driving behavior, Deep MEIRL-PPO outperforms MEIRL-PPO by 50.71% and 43.90% on the basis of the MAE and HD, respectively. Furthermore, it is discovered that Deep MEIRL-PPO accurately learns the behavior of human drivers avoiding potential conflicts when lines of sight are occluded. This research can contribute to aiding self-driving vehicles in learning human driving behavior and avoiding unforeseen risks.</p></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":null,"pages":null},"PeriodicalIF":2.8000,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Physica A: Statistical Mechanics and its Applications","FirstCategoryId":"101","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0378437124005612","RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PHYSICS, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
Driving behavior modeling is extremely crucial for designing safe, intelligent, and personalized autonomous driving systems. In this paper, a modeling framework based on Markov Decision Processes (MDPs) is introduced that emulates drivers’ decision-making processes. The framework combines the Deep Maximum Entropy Inverse Reinforcement Learning (Deep MEIRL) and a reinforcement learning algorithm-proximal strategy optimization (PPO). A neural network structure is customized for Deep MEIRL, which uses the velocity of the ego vehicle, the pedestrian position, the velocity of surrounding vehicles, the lateral distance, the surrounding vehicles’ type, and the distance to the crosswalk to recover the nonlinear reward function. The dataset of drone-based video footage is collected in Xi’an (China) to train and validate the framework. The outcomes demonstrate that Deep MEIRL-PPO outperforms traditional modeling frameworks (Maximum Entropy Inverse Reinforcement Learning (MEIRL) - PPO) in modeling and predicting human driving behavior. Specifically, in predicting human driving behavior, Deep MEIRL-PPO outperforms MEIRL-PPO by 50.71% and 43.90% on the basis of the MAE and HD, respectively. Furthermore, it is discovered that Deep MEIRL-PPO accurately learns the behavior of human drivers avoiding potential conflicts when lines of sight are occluded. This research can contribute to aiding self-driving vehicles in learning human driving behavior and avoiding unforeseen risks.
期刊介绍:
Physica A: Statistical Mechanics and its Applications
Recognized by the European Physical Society
Physica A publishes research in the field of statistical mechanics and its applications.
Statistical mechanics sets out to explain the behaviour of macroscopic systems by studying the statistical properties of their microscopic constituents.
Applications of the techniques of statistical mechanics are widespread, and include: applications to physical systems such as solids, liquids and gases; applications to chemical and biological systems (colloids, interfaces, complex fluids, polymers and biopolymers, cell physics); and other interdisciplinary applications to for instance biological, economical and sociological systems.