PHiD: Preserving human identity in pose-guided character animation

Abstract

Generating pose-guided human animation videos is a challenging task, particularly in maintaining consistent facial identity (ID) between the generated video and the reference image. Despite significant advancements in diffusion-based human animation models, existing methods, which mainly rely on basic conditioning mechanisms, often struggle with facial consistency and realism, especially when the face occupies a small region in the reference. This paper proposes a facial-area-aware approach, PHiD, designed to enhance facial ID similarity while ensuring strong structural and temporal coherence. Specifically, we propose a Pose-Driven Face Morphing module that leverages the 3D Morphable Model to synthesize proxy faces based on the reference ID and target pose, generating multi-view features to enhance temporal consistency. Additionally, we introduce a Masked Face Adapter (MFA) that embeds the proxy face and employs masked attention on facial regions to capture and refine localized facial features accurately. To enable effective training of MFA, we design a Facial ID-Preserving Loss that combines feature similarity, reconstruction, and pose consistency terms. Notably, our method demonstrates strong generalization capabilities and can be seamlessly integrated into existing pose-guided image-to-video models. Extensive experiments show that our approach outperforms baseline methods in generating human animation videos with improved facial consistency and similarity.