Panoptic-Deeplab-DVA: Improving Panoptic Deeplab with Dual Value Attention and Instance Boundary Aware Regression

Abstract

Panoptic DeepLab is a state-of-the-art framework that has showed good tradeoff between performance and complexity. In this paper, we focus on improving it to increase wide deployment of panoptic segmentation on mobile devices with low complexity. Specifically, we first present a novel Dual Value Attention (DVA) module to enable context information exchange between the semantic segmentation branch and the instance segmentation branch. Second, we further propose a new instance Boundary Aware Regression (iBAR) loss that assigns more emphasis on the instance boundary during instance regression. To assess the effectiveness of our proposed approach, we evaluate the performance on MSCOCO dataset for panoptic segmentation task, to show that our approach can improve upon the state-of-the-art Panoptic DeepLab with both the light-weight backbone network MobileNetV3 and the heavy-weight backbone network HRNetV2.