Multi-patch attention Transformer for multivariate long-term time series forecasting of TBM excavation parameters

Abstract

To address the research gap in multivariable long-term time series forecasting in the field of tunnel boring machine (TBM) and provide long-term insights for decision-making in TBM construction, this paper studies a novel Transformer-based forecasting model. Leveraging a multi-patch attention mechanism, the newly developed multi-patch attention Transformer (MPAT) model is designed to predict long-term trends of multiple TBM operation parameters. The innovation lies in finding the most relevant time delay series of the input series through autocorrelation calculation, and designing a multi-patch attention mechanism to replace the traditional attention mechanism of Transformer, so that the model can capture local and global information of the series and improve the accuracy of long-term prediction of high-frequency and weakly periodic TBM data. Experimental results have shown that MPAT model has a significant effect on capturing TBM data in terms of temporal dependencies. In a case study, we applied MPAT to the Rongjiang Guanbu Water Diversion Project in Guangdong Province and predicted four excavation parameters. The experimental results show that MPAT exhibits accurate predictive ability when the input length is 36 and the outputs are 12, 24, 48, and 72, respectively. In comparison with some state-of-the-art models, MPAT outperforms MSE by 19.1%, 23.6%, 36.4%, and 48.3%, respectively. We also discussed the impact of input length and the number of patches on performance, and found that each prediction length has the best input length corresponding to it, and longer inputs don’t represent more accurate predictions. The determination of the number of patches should also depend on the input length, as too many or too few patches can affect the capture of local information in the sequence.