Learning spatiotemporal dynamics with a pretrained generative model

Abstract

Reconstructing spatiotemporal dynamics with sparse sensor measurement is a challenging task that is encountered in a wide spectrum of scientific and engineering applications. The problem is particularly challenging when the number or types of sensors (for example, randomly placed) are extremely sparse. Existing end-to-end learning models ordinarily do not generalize well to unseen full-field reconstruction of spatiotemporal dynamics, especially in sparse data regimes typically seen in real-world applications. To address this challenge, here we propose a sparse-sensor-assisted score-based generative model (S3GM) to reconstruct and predict full-field spatiotemporal dynamics on the basis of sparse measurements. Instead of learning directly the mapping between input and output pairs, an unconditioned generative model is first pretrained, capturing the joint distribution of a vast group of pretraining data in a self-supervised manner, followed by a sampling process conditioned on unseen sparse measurement. The efficacy of S3GM has been verified on multiple dynamical systems with various synthetic, real-world and laboratory-test datasets (ranging from turbulent flow modelling to weather/climate forecasting). The results demonstrate the sound performance of S3GM in zero-shot reconstruction and prediction of spatiotemporal dynamics even with high levels of data sparsity and noise. We find that S3GM exhibits high accuracy, generalizability and robustness when handling different reconstruction tasks. Reconstructing and predicting spatiotemporal dynamics from sparse sensor data is challenging, especially with limited sensors. Li et al. address this by using self-supervised pretraining of a generative model, improving accuracy and generalization.