Monthly streamflow forecasting with temporal-periodic transformer

Abstract

Monthly streamflow forecasting is important for water resources planning and management in hydrology. In recent years, deep learning based data-driven approaches have received significant attention, especially the Long Short-Term Memory (LSTM) and the Transformer. Among the above two sorts of models for such a task, hardly any model considers the periodic information from the same month of different years directly. This periodic information is important for monthly streamflow forecasting and we propose a periodic attention mechanism to explore it in this paper. Specifically, we propose a novel Temporal-Periodic Transformer (TPT) model, which has temporal-periodic attention modules exploring the temporal information and the periodic information. As a comparison, the original Transformer-based streamflow forecasting model does not consider such periodic information explicitly. To show the performance of our TPT model, two datasets including the Catchment Attributes and Meteorology for Large-sample Studies in Australia (CAMELS-AUS) and a dataset from the Tangnaihai Hydrological Station located in Qinghai Province of China are employed in this paper. Our TPT model outperforms the benchmark Transformer model significantly, e.g., for Nash–Sutcliffe efficiency, the TPT model improves over the original Transformer-based model in 45.9% and furthermore its NSE achieves 0.9108 in Tangnaihai by pretraining in 20 selected basins in CAMELS-AUS. For monthly streamflow forecasting, the TPT model is a good choice.