Transformer network for time series prediction via wavelet packet decomposition

Abstract

Time series predictions are commonly used in the fields of energy, meteorology, and finance, among others. The accurate prediction of time series data is critical for making decisions and planning. In the real world, non-stationary time series data with statistical properties shift over time, making prediction more challenging. Although, conventional time series processing methods—such as multi-scale feature extraction or Transformer-based algorithms—produce superior prediction results, when dealing with data that contain more noise and outliers, the prediction ability of such methods can suffer. To address this problem, we proposed the WPFormer model, which incorporated time-frequency analysis into the Transformer architecture to increase the long-term series prediction accuracy. The model employed wavelet packet decomposition to identify and eliminate noise efficiently, increasing its immunity to interference. We evaluated WPFormer on four publicly available datasets and compared its performance against the Informer, LogTrans, Reformer, LSTMa, LSTNet, and DeepAR models using MSE and MAE metrics. On average, the WPFormer model surpassed the benchmark models by 16%.