Long-short Term Cross Echo State Network for time series forecasting task

Abstract

Investigating the dynamics of time series in nonlinear systems has become a prominent research focus in both theoretical and practical domains. Unveiling the intrinsic characteristics of nonlinear time series can significantly enhance the understanding and modelling of nonlinear systems. Among the various time prediction models, Reservoir Computing (RC) has garnered widespread attention due to its distinctive hidden layer architecture. The Echo State Network (ESN) is one of the most representative instances within the RC framework. However, most existing ESNs do not explicitly capture the fixed multi-scale dependencies in time series, and their short-term memory (STM) cannot meet the needs of specific time series. To address these limitations, this paper introduces a novel Echo State Network with a heterogeneous topology, named the Long-short Term Cross Echo State Network (LS-CrossESN). The overall architecture of this model consists of three different types of reservoirs in parallel. And it incorporates a heterogeneous topology structure known as the cross architecture, which merges those of the first reservoir with the state characteristics of the second reservoir, so that the information between the two reservoirs can be transmitted to each other. At the same time, a time-delay operator is inserted in the second reservoir, so that the fused characteristics would not be immediately input to the next layer but transmitted to the deep layer. In this way, the characteristics of input would not decay with the update of the layers. The structure of third reservoir captures the influence of recent historical memory through a specific sliding window technology, and finally the multi-scale states from each layer would be collected for combined prediction. To optimize parameters in this model, an Improved Salp Swarm Algorithm (ISSA) is proposed. The model was tested of eight datasets spanning three categories: Mackey-Glass series, Lorenz chaotic series, Sunspot series, airport temperature series, and two real network traffic datasets. The experimental results demonstrate that the STM of LS-CrossESN is significantly improved compared with Deep-ESN, LS-ESN, DATDR and ADRC. Across all eight datasets, the model exhibits robust performance in both one-step-ahead and multi-step predictions.