Reinforcement Learning for Freeway Lane-Change Regulation via Connected Vehicles

Abstract

Lane change decision-making is a complex task due to intricate vehicle-vehicle and vehicle-infrastructure interactions. Existing algorithms for lane-change control often depend on vehicles with a certain level of autonomy (e.g., autonomous or connected autonomous vehicles), which limits their applicability under low penetration rates of automated vehicles. To address this issue, this study proposes a lane-change regulation framework using multi-agent reinforcement learning (MARL) to improve freeway traffic efficiency via connected vehicles. Specifically, lane-change regulation signals (e.g., ‘left change allowed” or ‘right change disallowed”) are computed at a traffic management center and then broadcast to connected vehicles. Thus, human-driven vehicles remain uncontrolled, while connected vehicles follow the received regulation signals. Compared to vehicle-level maneuver control, this approach reduces communication and positioning requirements, avoids direct trajectory intervention, and can accommodate different levels of connectivity and autonomy. To further bridge the gap between theoretical macroscopic analysis and microscopic simulations, the proposed framework combines a microscopic simulation environment and a macroscopic lane-grid representation. The microscopic simulator is used to generate vehicle-level trajectories and execute the realized traffic dynamics, while the macroscopic framework aggregates microscopic vehicle information into lane-grid states, enabling low-cost control at the grid-agent level. Based on a multi-lane macroscopic traffic model represented by partial differential equations (PDEs), lane changes are interpreted as source-term exchanges between adjacent lanes, which can be regulated by the proposed lane-change regulation signals. Our model is evaluated in varied traffic scenarios and demand conditions using microscopic simulation. Experimental results demonstrate that the method improves overall traffic efficiency with limited additional energy consumption while maintaining driving safety.