Enhancing usability in face privacy protection via vision-language guided diffusion model

With the development of the Internet, a large number of images containing faces are widely shared on social media, leading to increased risks of face-based identity tracking and privacy breaches. Face de-identification serves as a privacy protection technique that conceals identifiable personal information in images. Recent advancements in generative model-based face de-identification methods have made progress in ensuring privacy while preserving image usability. However, challenges remain in enhancing the usability. Specifically, current methods often generate images with noticeable artifacts or struggle to preserve the original semantic information, which can hinder the practical applications in various computer vision tasks. In this paper, we propose a vision-language understanding-guided diffusion model for face de-identification. Our method incorporates a semantic preservation module and an identity protection module to guide the diffusion model in generating de-identified images. The semantic preservation module leverages a vision-language model to retain the sentence-level semantic information of the original image. The identity protection module perturbs the identity representation to ensure privacy. We train and evaluate our method on different datasets, and the experimental results demonstrate that, while ensuring privacy protection, our method not only surpasses existing methods in image quality but also outperforms them across multiple fine-grained utility tasks.