UniFormer: Consistency regularization-based semi-supervised semantic segmentation via differential dual-branch strongly augmented perturbations

Abstract

Consistency regularization is a common approach in the field of semi-supervised semantic segmentation. Many recent methods typically adopt a dual-branch structure with strongly augmented perturbations based on the DeepLabV3+ model. However, these methods suffer from the limited receptive field of DeepLabV3+ and the lack of diversity in the predictions generated by the dual branches, leading to insufficient generalization performance. To address these issues, we propose a novel consistency regularization-based semi-supervised semantic segmentation framework, which adopts dual-branch SegFormer models as the backbone to overcome the limitations of the DeepLabV3+ model, termed UniFormer. We present a Random Strong Augmentation Perturbation (RSAP) module to enhance prediction diversity between the dual branches, thereby improving the robustness and generalization performance of UniFormer. In addition, we introduce a plug-and-play self-attention module that can effectively model the global dependencies of visual features to improve segmentation accuracy. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on most evaluation protocols across the Pascal, Cityscapes, and COCO datasets. The code and pre-trained weights are available at: https://github.com/qskun/UniFormer.