A joint travel mode and departure time choice model in dynamic multimodal transportation networks based on deep reinforcement learning

Decision on travel choices in dynamic multimodal transportation networks is non-trivial. In this paper, we tackle this problem by proposing a new joint travel mode and departure time choice (JTMDTC) model based on deep reinforcement learning (DRL). The objective of the model is to maximize individuals travel utilities across multiple days, which is accomplished by establishing a problem-specific Markov decision process to characterize the multi-day JTMDTC, and developing a customized Deep Q-Network as the resolution scheme. To render the approach applicable to many individuals with travel decision-making requests, a clustering method is integrated with DRL to obtain representative individuals for model training, thus resulting in an elegant and computationally efficient approach. Extensive numerical experiments based on multimodal microscopic traffic simulation are conducted in a real-world network of Suzhou, China to demonstrate the effectiveness of the proposed approach. The results indicate that the proposed approach is able to make (near-)optimal JTMDTC for different individuals in complex traffic environments, that it consistently yields higher travel utilities compared with other alternatives, and that it is robust to different model parameter changes.