Dual-Branch Semantic Enhancement Network Joint With Iterative Self-Matching Training Strategy for Semi-Supervised Semantic Segmentation

Abstract

With the rapid development of deep learning, supervised training methods have become increasingly sophisticated. There has been a growing trend towards semi-supervised and weakly supervised learning methods. This shift in focus is partly due to the challenges in obtaining large amounts of labeled data. The key to semi-supervised semantic segmentation is how to efficiently use a large amount of unlabeled data. A common practice is to use labeled data to generate pseudo labels for unlabeled data. However, the pseudo labels generated by these operations are of low quality, which severely interferes with the subsequent segmentation task. In this work, we propose to use the iterative self-matching strategy to enhance the self-training strategy, through which the quality of pseudo labels can be significantly improved. In practice, we split unlabeled data into two confidence types, i.e., reliable images and unreliable images, by an adaptive threshold. Using our iterative self-matching strategy, all reliable images are automatically added to the training dataset in each training iteration. At the same time, our algorithm employs an adaptive selection mechanism to filter out the highest-scoring pseudo labels of unreliable images, which are then used to further expand the training data. This iterative process enhances the reliability of the pseudo labels generated by the model. Based on this idea, we propose a novel semi-supervised semantic segmentation framework called SISS-Net. We conducted experiments on three public benchmark datasets: Pascal VOC 2012, COCO, and Cityscapes. The experimental results show that our method outperforms the supervised training method by 9.3%. In addition, we performed various joint ablation experiments to validate the effectiveness of our method.