Self-Supervised Image Harmonization via Region-Aware Harmony Classification

Abstract

Image harmonization is a widely used technique in image composition, which aims to adjust the appearance of the composited foreground object according to the style of the background image so that the resulting composited image is visually natural and appears to be photographed. Previous methods are mostly trained in a fully supervised manner, while demonstrating promising results, they do not generalize well to complex unseen cases involving significant style and semantic difference between the composited foreground object and the background image. In this paper, we present a self-supervised image harmonization framework that enables superior performance on complex cases. To do so, we first synthesize a large amount of data with wide diversity for training. We then develop an attentive harmonization module to adaptively adjust the foreground appearance by querying relevant background features. To allow more effective image harmonization, we develop a region-aware harmony classifier to explicitly judge whether an image is harmonious or not. Experiments on several datasets show that our method performs favourably against previous methods. Our code will be made publicly available.