Retouched Face Image Quality Assessment Based on Differential Perception and Textual Prompt

Abstract

Face retouching involves using digital techniques to alter an individual’s appearance, commonly using in social media. However, excessively retouched face (RF) images can lead to issues such as unrealistic beauty standards and psychological stress. Therefore, it is crucial to develop a reliable quality assessment method for RF images. In this paper, we propose a novel network named DIRF-IQA for RF image quality assessment (IQA). DIRF-IQA mainly includes a parameter-shared image encoder, a text encoder, and three key components, namely the Differential Feature Attention Module (DFAM), the Text-image Interaction Module (TIM), and the Multi-scale Feature Fusion Module (MFFM). Specifically, the DFAM captures both local and global differences between original and retouched images by processing multi-scale features and utilizing cross-attention and self-attention blocks for differential perception. In the TIM, textual prompts summarizing retouching operations are encoded by a text encoder and integrated with differential features extracted by the DFAM to enhance the understanding of distortions in RF images. The MFFM then fuses these text-enhanced features across different layers and combines them with the global differential feature to predict the quality of the retouched images. We conduct extensive experiments on two RF IQA databases and the results demonstrate the superiority of DIDF-IQA compared to 12 state-of-the-art full-reference IQA methods in evaluating RF images.