Interactive Painting Volumetric Cloud Scenes with Simple Sketches Based on Deep Learning

Abstract

Synthesizing realistic clouds is a complex and demanding task, as clouds are characterized by random shapes, complex scattering and turbulent appearances. Existing approaches either employ two-dimensional image matting or three-dimensional physical simulations. This paper proposes a novel sketch-to-image deep learning system using fast sketches to paint and edit volumetric clouds. We composed a dataset of 2000 real cloud images and translated simple strokes into authentic clouds based on a conditional generative adversarial network (cGAN). Compared to previous cloud simulation methods, our system demonstrates more efficient and straightforward processes to generate authentic clouds for computer graphics, providing a widely accessible sky scene design approach for use by novices, amateurs, and expert artists.