Joint stage features modulation for progressive blind face restoration

A significant challenge for Blind Face Restoration (BFR) is to cope with the degraded information of unknown parameters in face images. The BFR method has evolved from non-prior to prior-based methods, but there are still some shortcomings. The quality of priors seriously affects the restoration results, especially in scenarios with severe degradation. Simultaneously encoding or modulating degraded images directly into the restoration process can introduce degraded information, leading to poor visual perception. Therefore, we propose a progressive restoration model with the joint stage features modulation, named JSFM-GAN. JSFM-GAN can be seen as having two stages. In the first stage, the LQ image is modulated with the facial resolution map to provide a rough structure for recovery. In the second stage, Joint Stage Feature Modulation (JSFM) utilizes the LQ images and stage features for joint modulation on multiple scales to balance fidelity and realism by combining clean spatial information of stage features and the tonal structure of LQ images. At the same time, the Up-Sampling Feature Supplement Block (UFSB) is used to reduce information loss due to channel fading and improve the network’s focus on face components and textures. In addition, we use the stage reconstruction loss and adjusted facial parsing maps to enhance the realism and symmetry of the generated results. Experiments with JSFM-GAN on synthetic and real-world datasets achieve good results, demonstrating the superior performance of our method.