Implementation of a WGAN-GP for Human Pose Transfer using a 3-channel pose representation

Abstract

The computational problem of Human Pose Transfer (HPT) is addressed in this paper. HPT in recent days have become an emerging research topic which can be used in fields like fashion design, media production, animation, virtual reality. Given the image of a human subject and a target pose, the goal of HPT is to generate a new image of the human subject with the novel pose. That is, the pose of the target pose is transferred to the human subject. HPT has been carried out in two stages. In stage 1, a rough estimate is generated and in stage 2, the rough estimate is refined with a generative adversarial network. The novelty of this work is the way pose information is represented. Earlier methods used computationally expensive pose representations like 3D DensePose and 18-channel pose heatmaps. This work uses a 3-channel colour image of a stick figure to represent human pose. Different body parts are encoded with different colours. The convolutional neural networks will now have to recognize colours only, and since these colours encode body parts, eventually the network will also learn about the position of the body parts.