Boosting semi-supervised regressor via confidence-weighted consistency regularization

Abstract

Semi-supervised regression aims to train a learner by utilizing both labeled and unlabeled data. Boosting is a popular approach in enhancing the performance of the base learner, which is often quite simple. One question is: can we boost different sophisticated semi-supervised regressors (SSRs) to provide better performance? In this paper, we propose a confidence-weighted consistency regularization (BS2C) algorithm to answer this question. First, we construct a neural network that works in parallel with an off-the-shelf SSR to provide pseudo-labels. In this way, the prediction ability of the regressor gradually shifts to the network. Second, we integrate supervised and consistency losses using a dynamic weighting strategy. Consequently, the impact of unlabeled data increases during training iterations. Third, we compute confidence and weights for pseudo-labels to guide the training. Therefore, the negative effect from network errors is reduced. Experiments were performed on fifteen real-world datasets using five popular SSRs, also in comparison with an existing boosting method. The results indicate that BS2C can boost SSRs in most cases, and superior than the counterpart. The source code is available at https://github.com/F1uency/BS2C.