A hybrid Capsule-Transformer Network for daily runoff forecasting

Abstract

Accurate prediction of daily runoff is crucial for flood prevention and water resource management. However, there exists complex interaction between periodic patterns and sudden fluctuations in hydrological processes. This makes accurate prediction challenging, especially in forecasting extreme events. Current mainstream deep learning methods struggle to simultaneously capture both local temporal dependencies and global temporal correlations. To address this challenge, CTNet (Capsule-Transformer Network) is proposed as a novel hybrid neural network architecture that combines the advantages of time capsule networks and transformers. Specifically, CTNet adopts dynamic routing policy to model different local capsule features, and self-attention mechanisms to learn long-term temporal dependencies, respectively. Furthermore, a cyclic embedding mechanism is proposed to assist in modeling temporal periodicity at different time scales. Extensive experiments was conducted on three datasets: the original Qingxi River basin dataset and two interpolation-enhanced datasets (DI-32 and DI-64). On the original dataset, the mean absolute error (MAE), root mean square error (RMSE), Nash–Sutcliffe efficiency (NSE), correlation coefficient (CC), and Willmott’s index (WI) values of CTNet reached 2.79, 10.65, 0.89, 0.945, and 0.971, respectively. It comprehensively outperforms current state-of-the-art models in both runtime and performance.