SCMNet: Shared Context Mining Network for Real-time Semantic Segmentation

Abstract

Different architectures have been adopted for realtime scene segmentation. A popular design is the multi-branch approach in which multiple independent branches are deployed at the encoder side to filter input images at different resolutions. The main purpose is to reduce the computational cost and handle high resolution. However, independent branches do not contribute in the learning process. To address this issue, we introduce a novel approach in which two branches at the encoder share their knowledge whilst generating the global feature map. At each sharing point, the shared features will go through a new effective feature scaling module, called the Context Mining Module (CMM), which will refine the shared knowledge before passing it to the next stage. Finally, we introduce a new multidirectional feature fusion module which fuses deep contextual features with shallow features successively with accurate object localization. Our novel scene parsing model, termed SCMNet, produces 66.5% validation mIoU on the Cityscapes dataset and 78.6% on the Camvid dataset whilst having only 1.2 million parameters. Furthermore, the proposed model can efficiently handle higher resolution input images whilst having low computational cost. Our proposed model produces state-of-the-art results on Camvid.