A reinforcement learning-based dynamic multi-objective optimization approach for integrated timetabling and vehicle scheduling

Abstract

Dynamic integrated timetabling and vehicle scheduling (D-ITVS) is essential for mitigating the negative impacts of service disruptions. It involves multiple rescheduling stages, with inherent optimization similarities across these stages. However, existing optimization approaches for the D-ITVS problem have not systematically exploited these similarities, overlooking the potential for decision knowledge from previous stages to inform the current stage. To address this gap, this paper proposes a reinforcement learning-based dynamic multi-objective optimization approach (RL-DMOA), which focuses on transferring decision knowledge between rescheduling stages. This approach models the optimization process of each rescheduling stage in the D-ITVS problem as a Markov decision process, incorporating a state space with vehicle information, action space for vehicle assignment, and a multi-objective reward function. A multi-objective deep reinforcement learning (M-DRL) agent is employed within the RL-DMOA to select actions based on the state at each decision point. The agent is constructed on a multi-objective deep Q-learning network (M-DQN), with a Q-value adjustment layer incorporated to prevent the selection of invalid actions. To select optimal actions while balancing the conflicts among multiple objectives, the M-DRL agent applies a non-dominated sorting selection strategy. Experimental results demonstrate that the proposed RL-DMOA is capable of generating timetables and vehicle schedules with reduced costs, enhanced robustness, and improved convergence and diversity across all rescheduling stages. By balancing operational costs and passenger service quality, these improvements benefit transit operators, and during daily operations, passengers enjoy reduced travel costs and enhanced service reliability.