FRNS: A Dual-Strategy Framework for Fine-Grained Feature Refinement and Text Noise Suppression in Text-Based Person Retrieval

Abstract

Text-based person retrieval aims to retrieve images of a specific individual based on textual descriptions. Recent approaches concentrate on aligning global and local features across text and image modalities. However, these methods struggle to obtain precise fine-grained image features and often ignore the noise in textual descriptions due to the disruption from the images’ background and the unmatched words in text. To alleviate these problems, we propose the Fine-grained Feature Refinement and Text Noise Suppression (FRNS) framework. This innovative solution consists of two key components: the Fine-grained Feature Refinement (FFR) module, designed to enhance the precision of local image features through the extraction of foreground information and the removal of background noise; and the Text Noise Suppression (TNS) module, which effectively mitigates the impact of irrelevant or misleading textual information by identifying and masking unmatched words with high probability. The experimental results demonstrate the effectiveness of our FRNS method on three widely-used datasets: CUHK-PEDES (Li et al., 2017), ICFG-PEDES (Ding et al., 2021) and RSTPReid (Zhu et al., 2021). Our code will be available on GitHub at https://github.com/ShijuanHuang/FRNS.