Multi-cropping contrastive learning and domain consistency for unsupervised image-to-image translation

Abstract

Recently, unsupervised image-to-image (i2i) translation methods based on contrastive learning have achieved state-of-the-art results. However, in previous works, the negatives are sampled from the input image itself, which inspires us to design a data augmentation method to improve the quality of the selected negatives. Moreover, the previous methods only preserve the content consistency via patch-wise contrastive learning, which ignores the domain consistency between the generated images and the real images of the target domain. This paper proposes a novel unsupervised i2i translation framework based on multi-cropping contrastive learning and domain consistency, called MCDUT. Specifically, the multi-cropping views are obtained with the aim of further generating high-quality negative examples. To constrain the embeddings in the deep feature space, a new domain consistency loss is formulated, which encourages the generated images to be close to the real images. In many i2i translation tasks, this method achieves state-of-the-art results, and the advantages of this method have been proven through extensive comparison experiments and ablation research. The code of MCDUT is available at https://github.com/zhihefang/MCDUT.