InpaintingPose: Enhancing human pose transfer by image inpainting

Abstract

Human pose transfer involves transforming a human subject in a reference image from a source pose to a target pose while maintaining consistency in both appearance and background. Most existing methods treat the appearance and background in the reference image as a unified entity, which causes the background to be disrupted by pose transformations and prevents the model from focusing on the complex relationship between appearance and pose. In this paper, we propose InpaintingPose, a novel human pose transfer framework based on image inpainting, which enables precise pose control without affecting the background. InpaintingPose separates the background from the appearance, applying transformations only where necessary. This strategy prevents the background from being affected by pose transformations and allows the model to focus on the coupling between appearance and pose. Additionally, we introduce an appearance control mechanism to ensure appearance consistency between the generated images and the reference images. Finally, we propose an initial noise optimization strategy to address the instability in generating human images with extremely bright backgrounds. By decoupling appearance and background, InpaintingPose can also recombine the appearance and background from different reference images to produce realistic human images. Extensive experiments demonstrate the effectiveness of our method, achieving state-of-the-art FID scores of 4.74 and 26.74 on DeepFashionv2 and TikTok datasets, respectively, significantly outperforming existing approaches.