CSGN:CLIP-driven semantic guidance network for Clothes-Changing Person Re-Identification

Abstract

Clothes-Changing Person Re-identification (CCReID) aims to match identities across images of individuals in different attires. Due to the significant appearance variations caused by clothing changes, distinguishing the same identity becomes challenging, while the differences between distinct individuals are often subtle. To address this, we reduce the impact of clothing information on identity judgment by introducing linguistic modalities. Considering CLIP’s (Contrastive Language-Image Pre-training) ability to align high-level semantic information with visual features, we propose a CLIP-driven Semantic Guidance Network (CSGN), which consists of a Multi-Description Generator (MDG), a Visual Semantic Steering module (VSS), and a Heterogeneous Semantic Fusion loss (HSF). Specifically, to mitigate the color sensitivity of CLIP’s text encoder, we design the MDG to generate pseudo-text in both RGB and grayscale modalities, incorporating a combined loss function for text-image mutuality. This helps reduce the encoder’s bias towards color. Additionally, to improve the CLIP visual encoder’s ability to extract identity-independent features, we construct the VSS, which combines ResNet and ViT feature extractors to enhance visual feature extraction. Finally, recognizing the complementary nature of semantics in heterogeneous descriptions, we use HSF, which constrains visual features by focusing not only on pseudo-text derived from RGB but also on pseudo-text derived from grayscale, thereby mitigating the influence of clothing information. Experimental results show that our method outperforms existing state-of-the-art approaches.