A multi-stage dynamic self-distillation network for industrial defect detection

Abstract

Self-distillation methods have made remarkable achievements in the field of object detection. However, in complex industrial defect detection, it usually faces the challenge of establishing a strong teacher network and effectively transferring knowledge. To address the above issues, a multi-stage dynamic self-distillation network (MSDNet) is designed, which can improve the accuracy of defect detection without adding additional parameters and computation time. Firstly, an auxiliary knowledge extraction (AKE) module is proposed to inject the physical characteristics and label attributes of industrial data into the teacher network, which provides powerful auxiliary information to enhance the performance of the teacher network. Secondly, a dynamic multi-dimensional distillation (DMD) module is designed, which enables the student network to learn important knowledge from the teacher network in three dimensions dynamically. Finally, a bi-directional multi-head distillation (BMD) module is designed, which can distill both foreground and background regions simultaneously to enhance the understanding of key knowledge of industrial defects and complex industrial fields. The experimental results show that the proposed method outperforms existing methods in defect detection (mean precision: MFL-DET: 3.1%; NEU-DET: 2.3% ; GC10-DET: 4.5%) and does not require additional parameter calculations.