Knowledge Transfer for Semantic Segmentation Based on Feature Aggregation

Abstract

In recent years, deep neural network has achieved high accuracy in the field of image recognition, but the consumption of resource is high. Inspired by collaborative learning approaches and the knowledge distillation technique, this study proposes a knowledge transfer method for semantic segmentation based on feature aggregation. In this research, the original student network is applied to generate the auxiliary teacher network, and share the information learned from the network by establishing dense feature connections between the two networks, which are trained simultaneously. Any one of these networks has access to information that is not available to the individual networks. In addition, in order to increase the degree of collaboration, this paper proposes two methods for establishing connections between the teacher network and the student network. The first method is a dense feature connection between networks of the same layer, and the second method is a dense feature connections between multi-layer networks. The approaches proposed above are validated on the train of Unet, and the experimental results show that the knowledge transfer approach of shared feature aggregation has better performance than the traditional single network.