Cvstgan: A Controllable Generative Adversarial Network for Video Style Transfer of Chinese Painting

Abstract

Style transfer aims to apply the stylistic characteristics of a reference image onto a target image or video. Existing studies on style transfer suffer from either fixed style without adjustability or unclear stylistic patterns in output results. Moreover, concerning video style transfer, issues such as discontinuity in content and time, flickering, and local distortions are common. Current research on artistic image style transfer mainly focuses on Western painting. In view of the differences between Eastern and Western painting, the existing methods cannot be directly applied to the style transfer of Chinese painting. To address the aforementioned issues, we propose a controllable style transfer method based on generative adversarial networks. The method operates directly in the feature space of style and content domains, synthesizing target images by merging style features and content features. To enhance the output stylization effect of Chinese painting, we incorporate stroke constraints and ink diffusion constraints to improve the visual quality. To mitigate issues such as blank spaces, highlights, and color confusion resulting in flickering and noise in Chinese painting style videos, we propose a flow-based stylized video optimization strategy to ensure consistency in content and time. Qualitative and quantitative experimental results show that our method outperforms state-of-the-art style transfer methods.