Large language models as spatiotemporal graph learning enhancers for large-scale traffic forecasting

Abstract

Understanding the traffic dynamics in spatial and temporal dimensions is essential to network-wide forecasting. Spatiotemporal graph (STG)-based prediction emerges as a promising method by integrating graph and temporal neural networks. Inspired by the extensive knowledge of large language models (LLMs), this paper leverages their understanding on traffic phenomena to enhance spatiotemporal forecasting. The LLMs are regard as general knowledge identifiers to recognize traffic patterns and underlying factors as prior knowledge, which is further vectorized based on a language model. An attention-based module is developed to incorporate the vectorized knowledge into STG models. The proposed framework was applied on a real-world traffic dataset, with multiple LLMs, STG models, and prediction horizons to evaluate the effects of LLM-identified knowledge on prediction accuracy and training efficiency. The incorporated knowledge significantly enhances comparatively weaker STG predictors over a relatively long horizon, especially in rush hours. It also leads to notable acceleration in STG training.