Solving multiscale dynamical systems by deep learning

Abstract

Multiscale dynamical systems, modeled by high-dimensional stiff ordinary differential equations (ODEs) with wide-ranging characteristic timescales, arise across diverse fields of science and engineering, but their numerical solvers often encounter severe efficiency bottlenecks. This paper introduces a novel DeePODE method, which consists of an Evolutionary Monte Carlo Sampling method (EMCS) and an efficient end-to-end deep neural network (DNN) to predict multiscale dynamical systems. The method's primary contribution is its approach to the “curse of dimensionality”– the exponential increase in data requirements as dimensions increase. By integrating Monte Carlo sampling with the system's inherent evolutionary dynamics, DeePODE efficiently generates high-dimensional time-series data covering trajectories with wide characteristic timescales or frequency spectra in the phase space. Appropriate coverage on the frequency spectrum of the training data proves critical for data-driven time-series prediction ability, as neural networks exhibit an intrinsic learning pattern of progressively capturing features from low to high frequencies. We validate this finding across dynamical systems from ecological systems to reactive flows, including a predator-prey model, a power system oscillation, a battery electrolyte thermal runaway, and turbulent reaction-diffusion systems with complex chemical kinetics. The method demonstrates robust generalization capabilities, allowing pre-trained DNN models to accurately predict the behavior in previously unseen scenarios, largely due to the delicately constructed dataset. While theoretical guarantees remain to be established, empirical evidence shows that DeePODE achieves the accuracy of implicit numerical schemes while maintaining the computational efficiency of explicit schemes. This work underscores the crucial relationship between training data distribution and neural network generalization performance. This work demonstrates the potential of deep learning approaches in modeling complex dynamical systems across scientific and engineering domains.