GAN vs Diffusion: Instance-Aware Inpainting on Small Datasets

Abstract

Instance-aware inpainting is a crucial task in many fields such as fashion, entertainment, and photography. However, developing effective instance-aware inpainting methods that can generalize to various target instances is a significant challenge when large-scale datasets are not available. In this study, we compare the performance of two state-of-the-art approaches for instance-aware inpainting, namely instance-aware GAN (InstaGAN) and RePaint, a denoising diffusion probabilistic model, using small datasets. We chose these methods for comparison as GANs are widely used for image generation, while diffusion-based methods are gaining popularity for their ability to generate high-quality images. Our experiments show that RePaint outperforms InstaGAN in small-scale instance-aware inpainting tasks. RePaint utilizes a diffusion process that models the image pixel values as a random walk, which effectively removes noise and provides better results than InstaGAN's instance-aware GAN approach. The diffusion process also enables RePaint to handle a wide range of noise distributions, making it more versatile for inpainting tasks. Our results provide quantitative evidence that RePaint outperforms InstaGAN in small-scale instance-aware inpainting tasks, with a lower FID score and LPIPS score. These findings emphasize the importance of selecting the appropriate model for a given dataset and task.