Diffinformer: Diffusion informer model for long sequence time-series forecasting

Abstract

Long sequence time-series forecasting (LSTF) is a significant research area with wide-ranging applications in energy, transportation, meteorology, and finance. Current methods primarily rely on statistical machine learning and deep neural network techniques to model historical time series for long-term forecasting. In recent years, Transformer-based models have demonstrated outstanding performance in forecasting, but their high computational costs limit their application. The Informer model addresses issues of high computational complexity and the management of long sequence inputs and outputs. However, existing models still face prediction bottlenecks under limited computational resources. The powerful generative capability of diffusion models can significantly enhance time series forecasting. We propose the Diffinformer model, which utilizes generative models for forecasting. Specifically, it combines conditional diffusion models with the ProbSparse self-attention distilling mechanism of Informer and incorporates the output of the diffusion model into the decoder to capture distant dependencies of observations from the perspective of dynamic systems. Comprehensive experimental results across five large-scale datasets demonstrate that Diffinformer improves predictive accuracy and outperforms corresponding baselines, offering a novel solution to the LSTF problem.