Evaluating the FixMatch Semi-Supervised Algorithm for Unbalanced Image Data

Abstract

In recent years there has been a major rise in interest in the field of semi-supervised deep learning with newer techniques being proposed which rapidly push the state-of-the-art. Most techniques, however, use balanced benchmarking datasets such as CIFAR and SVHN and therefore do not translate into real life applications, many of which involve highly imbalanced datasets. An investigation was conducted into the performance of the FixMatch algorithm when trained on unbalanced benchmarking datasets and a real world dataset. Three different entropy-based distributions of imbalance, with the proportion of labeled samples varied from 80% to 40%, were applied and compared to a baseline which was computed on uniformly balanced data. An increase in error rate is noted for the imbalanced datasets with larger errors seen in cases where there are a greater number of minority classes. Indeed, the distribution containing the most minority classes showed the maximum drop in performance with a mean error rate increase of 12.67% compared to the uniform baseline.