Controllable image generation and manipulation

Abstract

Recent years have witnessed an unprecedented interest in developing Deep Learning methodologies for the generation of images and image sequences that are hardly distinguishable to the human eye from real ones. A major issue in this field is how the generation can be easily controlled. In this talk we will focus on some of our recent works in generative models that are primarily aimed at controllable generation. We will first present unsupervised methods for learning non-linear paths in the latent spaces of Generative Adversarial Networks such that following different paths lead to different types of changes (e.g., removing the background, changing head poses, or facial expressions) in the resulting images [4]. Subsequently, we will present a method that allows local editing by finding a Parts and Appearances decomposition in the GAN latent space [2]. Then, we will present recent works on reenactment [1], where the goal is to transfer the facial activity (pose, expressions, speech) of a certain person to another one, and recent works in which supervision for generation comes from language models [3]. Finally, we will touch on the technical challenges ahead, as well on the challenges that this creates in spreading misinformation.