Maximizing the Generative Performance of Echo State Networks Using the Particle Swarm Optimization

This work reveals the hidden potential of two Echo State Networks (ESNs) in time-series generation tasks. The first system is a typical ESN with all-to-all connection weights. The latter possess sparse reservoir's connectivity and a limited number of input-output connections, or an Economy ESN (EcoESN). The standard Particle Swarm Optimization (PSO) is adopted to adjust the connection weight between the input and the reservoir. Two experiments regard the variation of the PSO training, and the rate of the input-output connection is conducted. The results confirm that optimizing the input connection of the ESN can boost the generative performance of the two networks. However, the EcoESN gains better benefit from the optimization and can surpass the fully connected in most of the tests. Furthermore, EcoESN with a low input-output rate of 0.1 can outperform the use of higher ones. This finding could shed insight into the construction of a lightweight and accurate generative model.