Multi-level Priors-Guided Diffusion-based Remote Sensing Image Super-Resolution

Abstract

Recently, diffusion models have achieved advancements in natural image super-resolution (SR) tasks, overcoming some issues posed by traditional approaches, e.g., performance limitations in CNN-based and Transformer-based approaches, as well as instable training and mode collapse in GAN. However, despite these advancements, existing diffusion-based SR methods fail to perform well for remote sensing images. Current diffusion-based super-resolution techniques face two key challenges: (1) A jeopardy to the generative prior arises due to the necessity of training from scratch, which can lead to suboptimal performance. (2) A loss of fidelity occurs due to the limited priors in SR models, which only take the low-resolution image as input. To deal with these challenges, we introduce a Multi-level Priors-Guided Diffusion-based Remote Sensing Image Super-Resolution Model (DLMSR) approach. In particular, we utilize a pre-trained stable diffusion model to maintain the generative prior captured in synthesis models, resulting in more stable and detailed outcomes. Furthermore, to establish comprehensive priors, we incorporate multimodal large language models (MLLMs) to capture diverse priors such as texture and content priors. Additionally, we introduce category priors by employing a category classifier to offer global and concise signals for precise reconstruction. Then, we devise a cascade prior fusion module and a class-aware encoder to integrate rich priors into the diffusion model. DLMSR is extensively evaluated on four publicly available remote sensing datasets, including AID, DOTA, DIOR, and NWPU-RESISC45, demonstrating consistent advantages over representative state-of-the-art methods. In particular, compared with StableSR, DLMSR achieves an average increase of 0.29 dB in PSNR and a decrease of 1.93 in FID across three simulated benchmarks, indicating enhanced reconstruction fidelity and perceptual quality. The source code and dataset links are publicly available at: https://github.com/lijing28/DLMSR.git.