Discrepancy-guided dual-branch semantic segmentation

In order to improve the feature extraction process in the semantic segmentation network and clarify the operating logic of the model, this paper splits the discriminative features according to the pixel's own information and the inter-pixel correlation information. Using two branch networks, relational branch and information branch to focus on the extraction of their own features, and using the fully convolutional segmentation head and the fully-connected segmentation head respectively to make the features more in line with the requirements of the branch itself. In order to make full use of the information contained in the discrepancy between the two branches, a Discrepancy-guided Fusion module is proposed at the feature level, in where the difference information between the features is used as a guide to promote the fusion of the features of the two branches. At the prediction level, with the help of Point-wise Discrepancy Cross Entropy loss, the prediction difference is used to determine the network's attention to each pixel. The effectiveness of the method is verified by extensive experiments on the Cityscapes dataset.