A Q-learning Foresighted Approach to Ego-efficient Lane Changes of Connected and Automated Vehicles on Freeways

Lane changes are a vital part of vehicle motions on roads, affecting surrounding vehicles locally and traffic flow collectively. In the context of connected and automated vehicles (CAVs), this paper is concerned with the impacts of smart lane changes of CAVs on their own travel performance as well as on the entire traffic flow with the increase of the market penetration rate (MPR). On the basis of intensive microscopic traffic simulation and reinforcement learning technique, an ego-efficient lane-changing strategy was first developed in this work to enable foresighted lane changing decisions for CAVs to improve their travel efficiency. The overall impacts of such smart lane changes on traffic flow of both CAVs and human-driven vehicles were then examined on the same simulation platform, which reflects a real freeway infrastructure with real demands. It was found that smart lane changes were beneficial for both CAVs and the entire traffic flow, if MPR was not more than 60%.