Bounding Box based Annotation Generation for Semantic Segmentation by Boundary Detection

Abstract

This paper proposes a new method to generate pseudo-annotations from manual bounding boxes for semantic segmentation. Different from traditional local data driven based methods such as Conditional Random Field (CRF) and GrabCut, we aim at using class-agnostic bounding box based segmentation models. To this end, we propose a new segmentation network, which formulates segmentation task as a sparse boundary point detection task rather than dense pixel label prediction task, and therefore can provide new type of pseudo-annotations. Furthermore, we detect object boundary based on direction, and use multiple directions to handle various shapes of objects. Moreover, we further enhance the pseudo generation by combining different types of segmentation masks. Classical Fully Convolutional Networks (FCN) network based on dense prediction is also modified to generate diverse foreground masks. A simple fusion method based on intersection operation is proposed to combine the two types of pseudo-annotations. We verify the effectiveness of our method on PASCAL VOC 2012 validation dataset. The mIoU value is 67.9%, which outperforms the state-of-the-art method by 1.1%.