ICTCAM: Introducing Convolution to Transformer-Based Weakly Supervised Semantic Segmentation

Abstract

Weakly supervised semantic segmentation(WSSS) is a challenging task, which only requires category information for segmentation prediction. Existing WSSS methods can be divided into two types: CNN-based and transformer-based, and the ways of generating pseudo labels are different. The former uses Class Activation Mapping(Cam)to generate pseudo labels, but there is a problem that the activated areas are concentrated in the most discriminative parts. The latter one choose to use attention map from the multi-head self-attention(MHSA) block, but there also exist the problems of significant background noise and incoherent object area. In order to solve the problems above, we propose ICTCAM to help transformer block obtain the ability of CNN, which include two modules named deeper stem(DStem) and convolutional feed-forward network(CFFN). The experiment results show that our modules have improved the performance of the network and achieve 69.9% mIoU, which is a new state-of-the-art performance on the PASCAL VOC 2012 dataset compared with similar networks.