EcoRouteQ: a reinforcement learning framework for green route recommendations

Abstract

This paper presents EcoRouteQ, a reinforcement learning-based framework designed to address the time-dependent green route recommendation problem (TDGRRP) in urban road networks. As urban congestion and environmental concerns grow, minimizing travel time and CO₂ emissions has become a critical goal for sustainable transportation. TDGRRP captures this challenge by seeking optimal routing decisions that account for time-varying traffic conditions and environmental impact, making the problem inherently dynamic and multi-objective.

The framework consists of three key components. First, in the preprocessing stage, raw global positioning system (GPS) data is segmented into trips and processed using map-matching to align trajectories with the road network. This is followed by the construction of a time-dependent dataset and corresponding road network graph based on extracted travel time, speed, and CO₂ emission data. A graph simplification algorithm is then applied to reduce network complexity while preserving critical topology, enabling more efficient learning. Second, the framework incorporates a policy training with adaptive reward system, which includes (i) an initialization module, (ii) a policy training loop employing Q-learning to learn optimal multi-objective routing policies over time-dependent networks, and (iii) a reward recalibration mechanism that refines immediate rewards based on the best historical trajectories at the end of each episode. The routing task is formulated as a Markov decision process (MDP) to enable sequential decision-making under dynamic traffic conditions. Together, these components address the curse of dimensionality, mitigate slow convergence, and reduce path deviations caused by dense reward structures. Finally, the eco-route prediction module uses the trained policy models to generate personalized, multi-objective routing decisions in response to user queries, balancing travel time and CO₂ emissions according to user preferences.

Extensive experiments on real-world datasets from Calgary and Edmonton demonstrate that EcoRouteQ reduces travel time by up to 30.63% and 30.74%, and CO₂ emissions by up to 31.15% and 30.02%, respectively. On synthetic simulations, the framework consistently out-performs Dijkstra, A^∗ and ADRL-TSC (a reinforcement learning-based baseline), achieving travel time improvements of 41.55%, 23.73%, and 10.00%, and CO₂ emission reductions of 61.54%, 28.57%, and 23.08%, respectively. These results highlight EcoRouteQ’s effectiveness in reducing travel and emissions costs by jointly optimizing multiple objectives under time-dependent traffic conditions.