SCSformer: cross-variable transformer framework for multivariate long-term time series forecasting via statistical characteristics space

Abstract

Deep learning-based models have emerged as promising tools for multivariate long-term time series forecasting. These models are finely structured to perform feature extraction from time series, greatly improving the accuracy of multivariate long-term time series forecasting. However, to the best of our knowledge, few scholars have focused their research on preprocessing time series, such as analyzing their periodic distributions or analyzing their values and volatility at the global level. In fact, properly preprocessing time series can often significantly improve the accuracy of multivariate long-term time series forecasting. In this paper, using the cross-variable transformer as a basis, we introduce a statistical characteristics space fusion module to preprocess the time series, this module takes the mean and standard deviation values of the time series during different periods as part of the model’s inputs and greatly improves the model’s performance. The Statistical Characteristics Space Fusion Module consists of a statistical characteristics space, which represents the mean and standard deviation values of a time series under different periods, and a convolutional neural network, which is used to fuse the original time series with the corresponding mean and standard deviation values. Moreover, to extract the linear dependencies of the time series variables more efficiently, we introduce three different linear projection layers at different nodes of the model, which we call the Multi-level Linear Projection Module. This new methodology, called the SCSformer, includes three innovations. First, we propose a Statistical Characteristics Space Fusion Module, which is capable of calculating the statistical characteristics space of the time series and fusing the original time series with a specific element of the statistical characteristics space as inputs of the model. Second, we introduce a Multi-level Linear Projection Module to capture linear dependencies of time series from different stages of the model. Third, we combine the Statistical Characteristics Space Fusion Module, the Multi-level Linear Projection Module, the Reversible Instance Normalization and the Cross-variable Transformer proposed in Client in a certain order to generate the SCSformer. We test this combination on nine real-world time series datasets and achieve optimal results on eight of them. Our code is publicly available at https://github.com/qiuyueli123/SCSformer.