Category-Guided Graph Convolution Network for Semantic Segmentation

Abstract

Contextual information has been widely used to improve results of semantic segmentation. However, most approaches investigate contextual dependencies through self-attention and lack guidance on which pixels should have strong (or weak) relationships. In this paper, a category-guided graph convolution network (CGGCN) is proposed to reveal the relationships among pixels. First, we train a coarse segmentation map under the supervision of the ground truth and use it to construct an adjacency matrix among pixels. It turns out that the pixels belonging to the same category have strong connections, and those belonging to different categories have weak connections. Second, a GCN is exploited to enhance the representation of pixels by aggregating contextual information among pixels. The feature of each pixel is represented by node, and the relationship among pixels is denoted by edge. Subsequently, we design four different kinds of network structures by leveraging the CGGCN module and determine the most accurate segmentation result by comparing them. Finally, we reimplement the CGGCN module to refine the final prediction from coarse to fine. The results of extensive evaluations demonstrate that the proposed approach is superior to the existing semantic segmentation approaches and has better convergence.