Contrastive attention and fine-grained feature fusion for artistic style transfer

Abstract

In contemporary image processing, creative image alteration plays a crucial role. Recent studies on style transfer have utilized attention mechanisms to capture the aesthetic and artistic expressions of style images. This method converts style images into tokens by initially assessing attention levels and subsequently employing a decoder to transfer the artistic style of the image. However, this approach often discards many fine-grained style elements due to the low semantic similarity between the original and style images. This may result in discordant or conspicuous artifacts. We propose MccSTN, an innovative framework for style representation and transfer, designed to adapt to contemporary arbitrary image style transfers as a solution to this problem. Specifically, we introduce the Mccformer feature fusion module, which integrates fine-grained features from content images with aesthetic characteristics from style images. Mccformer is utilized to generate feature maps. The target image is then produced by inputting the feature map into the decoder. We consider the relationship between individual styles and the overall style distribution to streamline the model and enhance training efficiency. We present a multi-scale augmented contrast module that leverages a substantial number of image pairs to learn style representations. Code will be posted on https://github.com/haizhu12/MccSTN