Fine-Grained Semantic Representation Learning for Visible–Infrared Person Re-Identification

Visible–infrared person re-identification (VI-ReID) faces great challenges due to the inherent cross-modality discrepancy. The key to reducing the discrepancy is to filter out the interference of modality information and project the pedestrian features in the two modalities into a shared feature space. However, previous works mainly focus on the application of high-level information and pay less attention to the middle-level features exploration. This limits the accuracy and generalization ability of cross-modality recognition. To address this shortcoming, we propose a novel fine-grained semantic representation learning (FSRL) network to explore the identity information in middle-level features for the VI-ReID task. Specifically, we first propose a plug-and-play modality normalization and compensation (MNC) module, which reduces the modality discrepancy while compensating for the missing identity information caused by modal elimination. Second, we propose an intermediate feature aggregation (IFA) module to obtain rich, fine-grained identity information in the middle layer, which guides the model to accurately extract more identity-related features for recognition. Finally, we also introduce the semantic-aligned feature learning (SAFL) module to further extract potential semantic part features from the feature map shared by the modalities to achieve cross-modality semantic alignment. Extensive experiments on the SYSU-MM01, RegDB DataSets (RegDB), and large-scale person re-identification dataset (LLCM) demonstrate the effectiveness and superiority of our proposed method.