Discriminative Candidate Selection for Image Inpainting Applications to the Fine Arts

Abstract

Within the field of Cultural Heritage, image inpainting is a conservation process that fills in missing or damaged parts of an artwork to present a complete image. Multi-modal diffusion models have brought photo-realistic results on image inpainting where content can be generated by using descriptive text prompts. However, these models fail to produce content consistent with a particular painter’s artistic style and period, being unsuitable for the reconstruction of fine arts and requiring laborious expert judgement. Moreover, generative models produce many plausible outputs for a given prompt. This work presents a methodology to improve the inpainting of fine art by automating the selection process of inpainted candidates. We propose a discriminator model that processes the output of inpainting models and assigns a probability that indicates the likelihood that the restored image belongs to a certain painter.