TWIST: An Efficient Spatial—Temporal Transformer With Temporal Window and Sparse Attention for Traffic Forecasting

Accurate traffic prediction is crucial for intelligent transportation systems (ITS). However, recent advancements in deep learning have encountered significant challenges in effectively capturing long-range dependencies and modeling complex intervariable relationships, particularly under real-time processing constraints. These limitations primarily arise from the inherent tradeoff in current model architectures: while some are optimized for short-term forecasting with constrained receptive fields, others prioritize long-term prediction accuracy at the cost of computational efficiency. Moreover, the dynamic and intricate interactions among nodes present significant scalability challenges, increasingly hindering efficient modeling as the road network grows. To address these limitations, we propose temporal window and sparse spatial attention transformer (TWIST), a novel architecture specifically designed for traffic forecasting. The proposed framework incorporates two key innovations: 1) a trend-aware window attention mechanism in the temporal dimension that effectively and efficiently captures multiscale temporal dynamics and 2) a sparse attention mechanism in the spatial dimension that optimizes computational efficiency by selectively focusing on significant spatial nodes. Extensive experiments on eight real-world datasets demonstrate that TWIST consistently outperforms state-of-the-art methods across various scenarios, including regular, long-term, and large-scale traffic forecasting tasks, while maintaining competitive computational efficiency. Our code is publicly available at https://github.com/STGTraffic/TWIST.