Inter-image Token Relation Learning for weakly supervised semantic segmentation

In recent years, Vision Transformer-based methods have emerged as promising approaches for localizing semantic objects in weakly supervised semantic segmentation tasks. However, existing methods primarily rely on the attention mechanism to establish relations between classes and image patches, often neglecting the intrinsic interrelations among tokens within datasets. To address this gap, we propose the Inter-image Token Relation Learning (ITRL) framework, which advances weakly supervised semantic segmentation by inter-image consistency. Specifically, the Inter-image Class Token Contrast method is introduced to generate comprehensive class representations by contrasting class tokens in a memory bank manner. Additionally, the Inter-image Patch Token Align approach is presented, which enhances the normalized mutual information among patch tokens, thereby strengthening their interdependencies. Extensive experiments validated the proposed framework, showcasing competitive mean Intersection over Union scores on the PASCAL VOC 2012 and MS COCO 2014 datasets.