A diffusion model using semantic and sketch information for anomaly detection

Abstract

In anomaly detection, methods that employ diffusion models for anomaly localization and reconstruction have demonstrated significant achievements. However, these methods face challenges such as the misclassification of multiple types of anomalies and the inability to effectively reconstruct large-scale anomalies due to the absence of semantic and sketch information from the original images. To tackle these challenges, we propose a framework, A Diffusion Model using Semantic and Sketch Information for Anomaly Detection (DSAD), which includes a semantic and sketch-guided network (SSG), a pre-trained autoencoder, and Stable Diffusion (SD). Initially, within SSG, we introduce a Semantic $\&$ Sketch Feature Fusion Module to enhance the model’s comprehension of the original images and present a Multi-scale Feature Fusion Module to maximize reconstruction accuracy. Subsequently, we connect SSG with the denoising network in SD in order to guide the network in reconstructing anomalous regions. Experiments on MVTec-AD dataset demonstrate the effectiveness of our approach which surpasses the state-of-the-art methods. The dataset and code are available at https://github.com/QinLi-STUDY/DSAD/tree/master.