A transformer-based approach for traffic prediction with fusion spatiotemporal attention

Abstract

Accurate traffic data prediction is a crucial technology for data-driven intelligent transportation systems. This has an important impact on optimizing urban traffic management, travel efficiency, traffic experience, etc. Traffic flow prediction tasks primarily focus on mining dynamic spatiotemporal dependencies. Most existing Transformer-based methods and GNN-based methods have limitations in mining local-global spatiotemporal dependencies. To address this issue, we propose a novel traffic data prediction model called LGSTformer that can perceive local-global spatiotemporal dependencies. First, we construct an embedding layer that provides multiple types of embedding representations for the model by projecting spatiotemporal data and temporal and spatial information into different embeddings. Next, we design two modules to capture local-global temporal and spatial dependencies based on the naive spatiotemporal self-attention mechanism: the local-global temporal module and the local-global spatial module. The former incorporates multi-scale temporal convolutions to capture short-term temporal dependencies, and the latter incorporates dynamic-static graph convolutions to capture local spatial dependencies. Finally, to achieve effective fusion of local-global dependency information, a dual-path adaptive gated fusion layer based on a gating mechanism is introduced to attain adaptive fusion of information at different levels. Experimental results on four public real-world traffic datasets show that LGSTformer outperforms existing methods and has potential as an advanced solution for traffic flow prediction.