Forecasting chaotic time series: Comparative performance of LSTM-based and Transformer-based neural network

Abstract

The complexity and sensitivity to initial conditions are the main characteristics of chaotic dynamical systems, making long-term forecasting a significant challenge. Deep learning, however, is a powerful technique that can potentially improve forecasting in chaotic time series. In this study, we explored the performance of modern neural network architectures in forecasting chaotic time series with different Lyapunov exponents. To accomplish this, we created a robust dataset composed of chaotic orbits with Lyapunov exponents ranging from 0.019 to 1.253 and used state-of-the-art neural network models for time series forecasting, including recurrent-based and transformer-based architectures. Our results show that LSTNet presents the best results in one-step-ahead and the recursive one-step-ahead forecasting for the majority of the time series in our dataset, enabling the prediction of chaotic time series with high Lyapunov exponent. Additionally, we observed that the sensitivity to initial conditions and complexity still affects the performance of the neural networks, decaying predictive power in time series with larger Lyapunov exponent.