Incremental Learning for Defect Segmentation With Efficient Transformer Semantic Complement.

Abstract

In industrial scenarios, semantic segmentation of surface defects is vital for identifying, localizing, and delineating defects. However, new defect types constantly emerge with product iterations or process updates. Existing defect segmentation models lack incremental learning capabilities, and direct fine-tuning (FT) often leads to catastrophic forgetting. Furthermore, low contrast between defects and background, as well as among defect classes, exacerbates this issue. To address these challenges, we introduce a plug-and-play Transformer-based semantic complement module (TSCM). With only a few added parameters, it injects global contextual features from multi-head self-attention into shallow convolutional neural network (CNN) feature maps, compensating for convolutional receptive-field limits and fusing global and local information for better segmentation. For incremental updates, we propose multi-scale spatial pooling distillation (MSPD), which uses pseudo-labeling and multi-scale pooling to preserve both short- and long-range spatial relations and provides smooth feature alignment between teacher and student. Additionally, we adopt an adaptive weight fusion (AWF) strategy with a dynamic threshold that assigns higher weights to parameters with larger updates, achieving an optimal balance between stability and plasticity. The experimental results on two industrial surface defect datasets demonstrate that our method outperforms existing approaches in various incremental segmentation scenarios.