Multi-granularity confidence learning for unsupervised text-to-image person re-identification with incomplete modality

Abstract

Most existing text-to-image person re-identification (TIReID) methods assume fully labeled and complete data for both person images and text descriptions. However, this assumption is unrealistic in real-world scenarios due to high labeling costs and privacy concerns. This work addresses a rarely explored task, i.e., unsupervised TIReID with incomplete modality learning. Prior approaches primarily focus on completing missing modalities and learning cross-modal consistency using global features. However, relying solely on these features limits the ability of model to comprehensively represent identities. Local features, which capture fine-grained details of data patches, are essential for effective cross-modal learning but are often overlooked. To address these limitations, we introduce WIN, a framework comprising three key components: (1) The Reliable Feature Completion (RFC) mechanism uses reciprocal neighbor mining to reconstruct both global and local features, ensuring the extraction of the most valuable information from multiple perspectives. (2) The Dual-Modal Clustering (DMC) mechanism addresses intra-class variability caused by missing labels by jointly clustering pedestrian identities from both modalities and generating pseudo-identity labels for supervised training. (3) To mitigate the inevitable noise introduced by imperfect feature completion and pseudo-label generation, we propose a Multi-Granularity Confidence Learning (MCL) mechanism. MCL leverages both global and local feature representations to estimate confidence scores for each sample and adaptively adjusts the optimization strength based on these scores, robustly bringing semantically similar visual and textual features closer in the common embedding space. Extensive experiments demonstrate the superiority of our method across diverse settings with missing modalities and in open-world environments.