A multi-task Transformer with mixture-of-experts for personalized periodic predictions of individual travel behavior in multimodal public transport

Integrated multimodal public transport (PT) systems are reshaping urban mobility by providing personalized travel experiences tailored to individual users. A critical challenge in realizing personalized mobility is predicting users’ periodic travel behaviors to capture each user’s evolving travel preferences and patterns. Big data and AI have opened new opportunities to accurately predict individual travel behavior, which is a critical initial step toward effective planning of personalized mobility bundle subscriptions and improvement of mobility services. This study proposes a novel framework, PTBformer-MMoE, for personalized periodic prediction of individual travel behavior, specifically predicting each user’s monthly mode-specific travel frequency class (classification tasks) and each user’s monthly expected travel fare (regression task), using the user’s most recent travel records. Within the multi-gate mixture-of-experts (MMoE) framework, each expert network is realized by a PTBformer, and each gate determines the weighted contributions of expert outputs relevant to a specific task tower. The PTBformer integrates two key modules, i.e., a Multi-mode Transformer employing multi-feature self-attention for continuous time-series travel data; and an OD Transformer capturing OD-specific travel features with multi-OD self-attention. Evaluated on a multimodal (bus, rail, ferry, and tram) dataset with over 0.96 billion travel records of 1.58 million users in Queensland, Australia, during 01/2021$-$01/2023, the proposed PTBformer-MMoE demonstrates state-of-the-art performance in predicting each user’s monthly mode-specific travel frequency class and monthly expected travel fare compared to 9 baseline models, setting a new benchmark for individual travel behavior predictions. The predictive capabilities of PTBformer-MMoE demonstrate its significant potential for real-world applications such as personalized mobility subscriptions, targeted recommendations, and optimized demand management, ultimately paving the way toward data-driven and user-centric multimodal PT systems.