Common Methods of Image Panoptic Segmentation Based on Deep Learning

In recent years, with the rapid development of deep learning technology and its wide application in the field of computer vision, various image understanding tasks including semantic segmentation and instance segmentation have made great progress, and people's further demand for image understanding has spawned a more comprehensive task image panoptic segmentation. Image panoptic segmentation can be seen as the combination of semantic segmentation and instance segmentation. For uncountable object categories (called stuff), the pixel category is distinguished. For countable object categories (called things), not only the semantic category of the target is recognized, but also each instance is distinguished. This task can provide more comprehensive scene information, and can be widely used in the understanding of various natural scenes. This paper investigate the commonly used panoptic segmentation methods, including the basic shared feature extraction method, the information combination method between semantic segmentation and instance segmentation sub-tasks, and the learnable method to remove the overlap between instances. This paper also summarize the commonly used panoptic segmentation datasets and the evaluation metrics, then the experimental performance evaluation results of various methods on commonly used datasets are showed. Finally, this paper summarize the general direction of panoptic segmentation, and predict the future research direction.