PMformer: A novel informer-based model for accurate long-term time series prediction

Abstract

When applied to long-term time series forecasting, Informer struggles to capture temporal dependencies effectively, leading to suboptimal forecasting accuracy. To address this issue, we propose PMformer, a novel model based on Informer for long-term time series prediction. First, we introduce a probabilistic patch sampling attention mechanism that utilizes a patch-based strategy to compute attention scores within randomly selected sequence patches. This localized approach enhances the model's capability to capture local temporal dependencies, allowing it to better understand and process critical local features in time series while reducing computational complexity. Additionally, we propose a multi-scale scaling sparse attention technique that balances attention distribution by combining coarse- and fine-grained attention scores, thereby improving the model's ability to capture global sequence information. Finally, we design a dilated causal pooling layer and a multilayer perceptual cross self-attention decoder to further enhance the model's prediction accuracy by capturing key information in long-term correlations and precisely focusing on sequences. We conducted experiments on both multivariate and univariate time series forecasting tasks. The results show that PMformer outperforms six baseline models, including PatchTST and FEDformer, in terms of MAE and MSE metrics. This demonstrates its superior ability to capture temporal dependencies, achieving more accurate predictions.