Structured guided diffusion models for industrial defect image generation

Abstract

Industrial defect images exhibit distinct characteristics from natural images, including severe class imbalance and structured similarity and diversity. Current defect image generation methods often lack fine-grained control over defect elements and suffer from limited diversity. This paper presents the Structured Guided Diffusion Model (Structured-GDM) for generating high-quality defect images with independent control over three structured elements: normal backgrounds, defect classes, and defect shapes. Controllability enables the generation of high-diversity defect images by preserving normal background outlines with detailed variation, specifying defect classes and shapes, and guiding the generation of reasonable (single or combined) defects using prior or expert knowledge. The structured architecture separates the training and use of elemental diffusion, classification, and segmentation models in a building-block manner, offering improved flexibility and maintainability. Additionally, a multiple-class training scheme is proposed to train overall models for one-for-all multiple-class defect generation, which exploits the inter-class similarity of defects and simplifies implementation. Extensive experiments on multiple MVTec and NEU-DET demonstrate that the method achieves superior performance in both image quality metrics and down-stream tasks, while maintaining high diversity and structured controllability.