HiFTLLM-Rail: A Hierarchical Fine-Tuned LLM for Spatio-Temporal Forecasting in Urban Railway Transit Systems

Abstract

Accurate forecasting of railway transit dynamics, such as passenger flow, is crucial for optimizing scheduling, mitigating congestion, and enhancing operational resilience. Recently, large language models (LLMs) have shown potential for spatio-temporal forecasting because their pre-trained transformer backbones can provide transferable contextual sequence modeling. However, LLM-enabled rail transit forecasting still faces three core challenges. First, railway demand contains multi-scale temporal patterns, including local bursts, periodic commuting rhythms, and long-horizon variations that must be represented without losing station-specific dynamics. Second, rail systems are defined by structured topology and heterogeneous station roles, whereas LLM backbones are inherently sequential and lack graph inductive bias. Third, continuous spatio-temporal observations must be aligned with the hidden representation space of pre-trained LLMs, and the backbone must be adapted efficiently under limited transportation data and compute. To address these challenges, we propose HiFTLLM-Rail, a hybrid framework that integrates topology-aware spatio-temporal encoding with a Hierarchical Fine-Tuned (HiFT) LLM. A dual-branch encoder couples global graph-based diffusion with local temporal aggregation through a gated fusion mechanism, producing topology-aware station embeddings that are aligned to the LLM hidden space. The LLM backbone is then adapted via HiFT, which selectively updates its upper transformer layers and normalization parameters for efficient domain adaptation. Extensive experiments on three large-scale real-world datasets demonstrate that HiFTLLM-Rail consistently outperforms state-of-the-art graph-based, time-series, and LLM-based forecasting baselines across multiple prediction horizons. Ablation and model-analysis results further verify the effectiveness of topology-aware embedding construction, adaptive fusion, the pre-trained backbone, and hierarchical adaptation. Our code is available at https://github.com/KarryHuang12138/HiFTLLM-Rail-code