AMSFormer: A transformer with adaptive multi-scale partitioning and multi-level spectral filtering for time-series forecasting

Abstract

Time series forecasting is essential in numerous real-world scenarios, yet it remains a challenging task due to complex temporal dependencies and the coexistence of multi-scale, periodic, and non-periodic patterns. In this paper, we present an innovative approach called the Adaptive Multi-Scale Transformer (AMSFormer) to tackle these challenges. AMSFormer integrates both frequency-domain and time-domain information to model local and global features collaboratively. By leveraging Fourier transforms, AMSFormer adaptively partitions time series data into patches that align with the data’s intrinsic patterns, enabling dynamic and efficient processing. A convolutional attention mechanism is employed to extract fine-grained local features at multiple scales while maintaining low computational overhead. For global feature extraction, AMSFormer utilizes a hierarchical frequency-domain filter to isolate key periodic components and suppress noise, enhancing the stability and accuracy of global pattern modeling. Extensive experiments on several real-world benchmark datasets demonstrate that AMSFormer consistently outperforms state-of-the-art models, highlighting its robust generalization ability and wide applicability across various forecasting tasks.