Art creator: Steering styles in diffusion model

Abstract

Large-scale text-to-image (T2I) generative models are extensively used in the art and creative industries because of their remarkable capability in generating high-quality images. The generation of ideal images in a single attempt is nearly impossible, necessitating complex and precise post-image editing. However, stylization pose significant challenges in post-editing. In this context, we introduce the Art Creator, which facilitates style controls based on a simple description or a single image. Art Creator enables nuanced image style edits, alterations in painting materials, colors, and brushstrokes, and understanding of high-level attributes such as object shapes. Furthermore, we manually annotated and released a dataset named ChinArt, comprising over 20,000 eastern artworks, aiming to address the gap in the global art creation domain. We showcase the quality and efficiency of our method across various art style creations.