Shifting decision boundary against adversarial semantic attacks in Latent Diffusion Models

Abstract

Image inpainting and generation have witnessed significant progress due to Latent Diffusion Models (LDMs), which map the diffusion and denoising processes from pixel space to a low-dimensional latent space. However, denoising is prone to introducing considerable loss bias, which accumulates in the latent space and manifests as imperceptible adversarial perturbations in generated samples. Such perturbations degrade downstream inference accuracy and may even lead up to task failures. To address the aforementioned issues, we propose a defence method, Shifting Adversarial Semantic Decision Boundary (Shifting-ASDB), which fine-tunes perturbed regions by shifting the decision boundary inward and outward, thereby reducing the sensitivity of adversarial examples near the decision boundaries of ground-truth samples. The proposed Shifting-ASDB not only mitigates the accuracy degradation problem arising from adversarial semantic attacks but also maintains the diversity of outputs while avoiding conspicuous misclassification in semantic regions. Extensive experiments and ablation studies conducted on benchmark semantic datasets demonstrate that our proposed defence method achieves superior robustness and generalisability in generative tasks, highlighting the method’s effectiveness in mitigating privacy risks and security concerns associated with unauthorised images in practical applications. These contributions hold practical value in ensuring compliance with privacy standards in security-sensitive applications.