Fusion feature contrastive learning and supervisory regularization for weakly supervised semantic segmentation

Abstract

Weakly supervised semantic segmentation (WSSS) based on image-level labels is a challenging task. WSSS methods using image-level labels typically employ Class Activation Maps (CAM) as pseudo labels. However, many methods using Convolutional Neural Network (CNN) models are affected by their local perception capabilities, resulting in CAM that only distinguish the most salient object regions. To address this issue, building upon the Vision Transformer (ViT) model as the backbone, we design a Fusion Feature Contrastive Learning (FFCL) method that utilizes feature information relationships from ViT’s intermediate layer to guide the final layer’s feature information, improving the quality of CAM. Moreover, We also propose a Supervisory Regularization (SR) strategy that fully utilizes auxiliary CAM feature information to guide the final layer’s CAM, enhancing the completeness of the CAM activation areas. The experiments on the PASCAL VOC 2012 and MS COCO 2014 datasets show that our proposed method achieves prominent improvements.