Mining Implicit Relations Among Image Channels for Few-Shot Semantic Segmentation

Abstract

The goal of few-shot semantic segmentation (FSS) is to segment the foreground image of an unseen class in the query image by using a few labeled support images. Existing two-branch models mine support and query image information to improve segmentation results by employing support prototypes, calculating the similarity between support and query images, or fusing multi-scale features. Such methods only focus on the spatial information of the query image in the initial feature extraction and subsequent processes. Meanwhile, limited by the sample size, their ability to extract channel information is insufficient, thus leading to the information loss of the query image. To solve the issues, we propose an implicit channel relation based few-shot semantic segmentation method entitled MANGO. The implicit relation mining process is implemented after the initial feature extraction and before the two branches interact to fully mine the query image information. Specifically, the query channel features are taken as nodes to construct the graph structure to establish the relationship between nodes. The network motif is used to quantity the attribute features and structural features of nodes to enhance the relationship between channels. Finally, we aggregate the two features and mine the implicit relationship of nodes through graph representation learning. Experiments on PASCAL-5i and FSS-1000 datasets demonstrate that our proposed method outperforms the state-of-the-art methods.