MEHGNet: a multi-feature extraction and high-resolution generative network for satellite cloud image sequence prediction

Satellite cloud image sequences contain rich spatial and temporal information, and forecasting future cloud image sequences is of great significance for meteorological research. Traditional satellite cloud image prediction methods usually ignore nonlinear variations in cloud masses, leading to large errors in prediction results and low prediction efficiency. The use of existing video prediction methods for satellite cloud image sequence prediction also suffers from problems of blurred prediction images and the accumulation of sequence errors. To address these issues, we propose a Multi-feature Extraction and High-resolution Generative Network (MEHGNet) for the prediction of satellite cloud image sequences, which consists of an encoder, a translator, a decoder, and a generator. To learn the spatial features and spatiotemporal dependencies of cloud images, 2D convolution multi-head attention mechanisms and local residue connections are introduced to the encoder and decoder. The generator preserves detailed features and improves the resolution of the predicted images using the generative ability of generative adversarial networks. In addition, a motion-aware loss function is proposed to learn high-level features of motion variations among cloud image sequences. Experiments on satellite datasets demonstrate that the proposed method is superior compared to other prediction methods.