Precise occlusion-aware and feature-level reconstruction for occluded person re-identification

Abstract

Occluded person re-IDentification (re-ID) is a challenging task in surveillance scenarios that remains unresolved. To address it, existing methods primarily rely on auxiliary models, e.g. pose estimation, to explore visible parts by detecting human keypoints. However, these approaches inevitably encounter two issues: domain gap and information asymmetry. The former arises from pre-training auxiliary models on different domains, while the latter indicates that the occluded query has asymmetric valid cues compared to the holistic visible gallery. In this paper, we propose a novel Precise Occlusion-aware and Feature-level Reconstruction (POFR) network for occluded re-ID. POFR addresses the occlusion issue from two viewpoints: perceiving the occlusions other than visible human bodies and reconstructing the occluded parts at the feature level. The first perspective is achieved through occlusion-driven contrastive learning (OCL). OCL incorporates an occlusion generator capable of generating object and person-specific occlusions. Unlike previous coarse occlusions, our generator leverages segmented pedestrians and obstacles to generate realistic occlusions which are then used for contrastive learning. The second perspective is implemented through an occlusion-guided feature reconstruction (OFR) module. OFR initially learns an occlusion predictor to estimate the occlusion mask, which is subsequently utilized to recover features corresponding to the occluded regions. Benefiting from the occlusion generator, the occlusion predictor can be effectively supervised with the precise occlusion masks, thereby mitigating the domain gap problem. Additionally, the recovered features alleviate information asymmetry when matching an occluded query and a holistic gallery. Extensive experiments conducted on occluded, partial, and holistic datasets demonstrate the superior performance of our POFR over state-of-the-art methods. The source code will be made publicly available upon paper acceptance.