A semi-supervised learning for teacher-student model based on structural reparameterization

Abstract

Semi-supervised learning (SSL) has attracted much interest for its ability to increase model performance using unlabeled data in recent years. The mainstreaming SSL frameworks are built on the teacher-student model. However, the structural reparameterization (SR) model has not been thoroughly studied in the teacher-student model. The SR technology enhances model capacity and improves performance during the training phase. Therefore, we introduce the SR model in the SSL framework, where the SR model is adopted to construct the teacher-student model for image classification. The consistency regularization is applied to the teacher and student models, and the teacher model’s weights are updated based on the exponential moving average (EMA) strategy. To verify the effectiveness of our approach, we conduct experiments using the CIFAR-10 and Food-101 datasets. Compared to the supervised learning of the SR model, our SSL framework has achieved better performance on these datasets.