Unveiling authenticity with diffusion-based face retouching reversal

Abstract

Unveiling the real appearance of retouched faces to prevent malicious users from deceptive advertising and economic fraud has been an increasing concern in the era of digital economics. This article makes the first attempt to investigate the face retouching reversal (FRR) problem. We first build an FRR dataset, named deepFRR, by collecting 50,000 StyleGAN-generated high-resolution (1024 × 1024) facial images and retouching them via a commercial online API. Then, we present a novel diffusion-based FRR network (FRRffusion) for the FRR task. Our FRRffusion consists of a coarse-to-fine two-stage architecture: A diffusion-based Facial Morpho-Architectonic Restorer (FMAR) is constructed to generate the basic contours of low-resolution faces in the first stage, while a Transformer-based Hyperrealistic Facial Detail Generator (HFDG) is designed to create high-resolution facial details in the second stage. Tested on deepFRR, our FRRffusion surpasses the state-of-the-art image restoration method with 22%, 11%, 20%, and 6% performance improvement in SSIM, PSNR, VGGS, and CLIPS, respectively. Especially, the de-retouched images by our FRRffusion are visually much closer to the raw face images than both the retouched face images and those restored by the state-of-the-art, like GP-UNIT and Stable Diffusion, in terms of qualitative evaluation with 85 subjects. These results sufficiently validate the efficacy of our FRRffusion, bridging the gap between the FRR and generic image restoration tasks. The code is available at https://github.com/GZHU-DVL/FRRffusion.