Exploiting Generative Diffusion Prior With Latent Low-Rank Regularization for Image Inpainting

Abstract

Generative diffusion models have recently shown impressive results in image restoration. However, the predicted noise from existing diffusion-based methods may be inaccurate, especially when the noise amplitude is small, thereby leading to sub-optimal results. In this letter, an unsupervised diffusion model with latent low-rank regularization is proposed to alleviate this challenge. In particular, we first create a latent low-rank space using self-supervised learning for each degraded images, from which we derive corresponding latent low-rank regularization. This regularization, combining with observed prior information and smoothness regularization, guides the reserve sampling process, resulting in the generation of high-quality images with fine-grained textures and fewer artifacts. In addition, by utilizing the pre-trained unconditional diffusion model, the proposed model reconstructs the missing pixels in a zero-shot manner, which does not need any reference images for additional training. Extensive experimental results demonstrate that our proposed method is superior to the self-supervised tensor completion methods and representative diffusion model-based image restoration methods.