Fear-constrained personalized and anthropomorphic reinforcement learning for autonomous car-following

Abstract

Achieving safe, personalized and anthropomorphic driving performance remain challenges for autonomous driving especially in the car-following scenario. “NeuroAI”, which combined neuroscience, brain science and psychology with artificial intelligence (AI), has shown great potential to enhance the performance of AI systems. Drawing inspiration from “NeuroAI”, we present a fear-constrained personalized and anthropomorphic (FCPA) reinforcement learning (RL) for autonomous car-following. Firstly, the fear model of the ego vehicle driver in the car-following scenario is established. And then, the fear thresholds of the drivers with different driving styles are determined through analyzing the collected driving data. Finally, the FCPA-RL algorithm is proposed to realize safe, personalized and anthropomorphic autonomous car-following by keeping the fear within corresponding thresholds and designing the reward function based on the probability density functions (PDF) of time headway (THW). Through experimental tests, we demonstrate that FCPA-RL effectively enhances safety during training, ​achieves personalized and anthropomorphic autonomous car-following​, and ​exhibits robust generalization​ across diverse driving scenarios beyond existing approaches. Furthermore, the results also reveal that FCPA-RL not only ​learns human drivers’ behavioral characteristics​ but also ​has potential to surpass human-level driving performance.