Text-guided Attention Mechanism Fine-grained Image Classification

Abstract

Scene texts with explicit semantic information in natural images can provide important clues to solve the corresponding computer vision problems. In the text, we usually focus on using multimodal content in the form of visual and text prompts to solve the task of fine-grained image classification and retrieval. In this paper, graph convolution network is used to perform multimodal reasoning, and the features of relationship enhancement are obtained by learning the common semantic space between salient objects and texts found in images. By obtaining a set of enhanced visual and textual functions, the proposed model is highly superior to the existing technologies in two different tasks (fine-grained classification and image retrieval in contextual texts).