EGSA: Enhanced and Global Semantic Activation for Weakly Supervised Object Localization

Abstract

Weakly supervised object localization(WSOL) is a task that only uses image-level supervision information to locate objects. Traditional CNN-based methods always locate the most discriminative regions of objects and cannot well balance the accuracy of classification and localization. To solve this problem, we propose an enhanced and global semantic activation(EGSA) method based on the vision transformer model. We first use an attention reassign module to get a comprehensive attention map that contains the correlation between each image patch and the global dependency of the class token. Then a mask selection module that generates a mask map by comparing with mask threshold is proposed to obtain the token feature map of the non-discriminative object region. By coupling the above two maps and combining it with a semantic aware map contains the information of class token, the final localization map with enhanced and global semantic activation can be built. And experiments on two common benchmark datasets CUB-200-2011 and ILSVRC demonstrate the efficiency of our method.