VTformer: a novel multiscale linear transformer forecaster with variate-temporal dependency for multivariate time series

Abstract

Recently, the prosperity of linear models has raised questions about capturing the sequential capabilities of Transformer forecasters. Although the latest Transformer-based studies have alleviated some of these concerns, the limited information utilization still constrains the model’s comprehensive exploration of complex dependencies, as these forecasters often prioritize global dependence on time stamps and overlook correlations between different variates. To this end, we reflect on the competence of Transformer components and present an efficient lightweight Transformer forecaster named VTformer. Concretely, a Transformer with multiscale linear attention is constructed to mine the global variate correlation and long-term temporal dependence of time series data in parallel, providing multifaceted dynamics for the downstream self-attention mechanism. Moreover, a novel adaptive fusion method is designed to propagate complementary information from the perspective of variate and temporal to promote prediction. Extensive experiments on eight real-world datasets demonstrate that VTformer outperforms state-of-the-art models in long-term Multivariate Time Series Forecasting (MTSF) tasks, thereby advancing the accuracy and efficiency of Transformers.