Multi-layer Feature Fusion Method with Fewer Connections for Fast Semantic Segmentation

Abstract

Feature fusion of spatial and semantic information is important to achieve high-performance semantic segmentation. However, fast semantic segmentation demands low computational complexity and challenges researchers to design structures efficiently. In recent years, Neural Network Architecture Search (NAS) has achieved better results in automatic network design. For lower computational complexity, we propose a multi-layer feature fusion with fewer connections in search space and add an improved penalty term for the loss function of the search algorithm to decrease the number of feature fusion connections. Based on the proposed multi-layer feature fusion method, we search the two-branch semantic segmentation model using the search algorithm reported by Gao's MTL-NAS. The experimental results tested on the Cityscapes dataset show that the searched module can improve the accuracy. For FastSCNN, ContextNet and BiSeNet, the mIoU improvement is 2%, 2.5% and 1%, respectively. The searched module is also more efficient than the densely connected structure.