Optimized Echo State Network based on PSO and Gradient Descent for Choatic Time Series Prediction

Abstract

Echo State Network (ESN), as a paradigm of Reservoir Computing (RC), refers to a well-known Recurrent Neural Network (RNN). Its randomly generated reservoir represents the main reason for its ability of rapid learning. Nevertheless, designing a reservoir for a specific role constitutes a difficult task. To resolve the challenge of the reservoir structure design, in this paper, a new combination of two optimization methods, Particle Swarm Optimization (PSO) and Stochastic Gradient Descent (SGD), have been proposed to reach a higher performance. The resulted model was tested using Mackey Glass and NARMA 10 benchmarks. The experimentations proved that the suggested PSO-SGD-ESN model performs well in time series prediction tasks and outperforms the original one.