Curriculum learning with class-label composition for weakly supervised semantic segmentation

Abstract

Weakly Supervised Semantic Segmentation (WSSS) aims to build a segmentation network using only weak labels. In WSSS training using image-level labels, a classifier is trained with multi-labeled images, as the task assumes the presence of multiple classes. The classifier plays a crucial role due to its impact on the quality of the derived pseudo-masks. However, training the classifier with the multi-labeled images presents two following challenges: (1) The presence of frequently co-occurring classes (e.g. chair and dining table) introduces a spurious correlation, making it difficult for the classifier to determine the location of each class. (2) Such multi-labeled datasets often exhibit imbalanced class distributions, which can create challenges during the training process. To tackle these issues, we propose a curriculum learning strategy based on the length and frequency of class-label composition. This strategy gradually reduces the influence of images with spurious correlation between classes and ensures that classes with fewer images appear more frequently during training. Our extensive experiments demonstrate that, when applied to eight WSSS methods, our curriculum strategy consistently enhances the quality of the pseudo-labels and segmentation performances, and also reduces the required computational resources for training.