Improving long-term autoregressive spatiotemporal predictions: A proof of concept with fluid dynamics

Abstract

Data-driven approaches have emerged as a powerful alternative to traditional numerical methods for forecasting physical systems, offering fast inference and reduced computational costs. However, for complex systems and those without prior knowledge, the accuracy of long-term predictions frequently deteriorates due to error accumulation. Existing solutions often adopt an autoregressive approach that unrolls multiple time steps during each training iteration; although effective for long-term forecasting, this method requires storing entire unrolling sequences in GPU memory, leading to high resource demands. Moreover, optimizing for long-term accuracy in autoregressive frameworks can compromise short-term performance. To address these challenges, we introduce the Stochastic PushForward (SPF) training framework in this paper. SPF preserves the one-step-ahead training paradigm while still enabling multi-step-ahead learning. It dynamically constructs a supplementary dataset from the model’s predictions and uses this dataset in combination with the original training data. By drawing inputs from both the ground truth and model-generated predictions through a stochastic acquisition strategy, SPF naturally balances short- and long-term predictive performance and further reduces overfitting and improves generalization. Furthermore, the training process is executed in a one-step-ahead manner, with multi-step-ahead predictions precomputed between epochs-thus eliminating the need to retain entire unrolling sequences in memory, thus keeping memory usage stable. We demonstrate the effectiveness of SPF on the Burgers’ equation and the Shallow Water benchmark. Experimental results demonstrated that SPF delivers superior long-term accuracy compared to autoregressive approaches while reducing memory consumption. This positions SPF as a promising solution for resource-constrained environments and complex physical simulations.