Characterizing the edge of chaos in echo state networks

Abstract

Chaos theory examines how simple deterministic rules can produce unpredictable yet highly structured dynamics due to their extreme sensitivity to initial conditions. In reservoir computing, and particularly in Echo State Networks (ESNs), operating at the so-called “edge of chaos” has been empirically shown to maximize memory capacity and computational richness; however, a rigorous characterization of this critical regime has remained elusive. Here, we address this gap by combining propagation-of-chaos Dynamical Mean-Field Theory (DMFT) with infinite-dimensional ergodic and multiplicative-ergodic theorems and sharp spectral-gap and minorization estimates to establish, for continuous-time ESNs, the almost-sure existence of a unique critical gain $g_c$ at which the maximal Lyapunov exponent $\Lambda \left(g\right)$ crosses zero. We derive an exact one-dimensional DMFT formula $\Lambda \left(g\right)=lng+E_{{\bar{\mu }}_g}[ln\left|{\varphi }^{\prime }\left(X\right)\right|]$, prove that it admits a single zero, and validate $g_c$ empirically via six independent diagnostics—spectral scaling, long-range correlations, fractal attractor dimension, phase-space geometry, invariant-measure statistics, and spatio-temporal coherence. Our results provide a theoretical foundation for edge-of-chaos ESNs, illuminating why marginally stable reservoirs yield optimal performance and laying the theoretical groundwork for integrating chaotic dynamics into modern machine learning architectures.