Image segmentation via two-step deep variational priors

Abstract

This paper proposes an iterative deep variational approach for image segmentation in a fusion manner: it is not only able to realize selective segmentation, but can also alleviate the issue of parameter/initialization dependency. Moreover, it possesses a refinement process designed to handle challenging scenarios, such as images containing obscured, damaged, or absent objects, or those with complex backgrounds. Our proposed approach consists of two main procedures, i.e., selective segmentation and shape transformation. The first procedure works as a stem in a totally unsupervised way. A convolutional neural network (CNN) based architecture is properly incorporated into the selective weighting constrained variational segmentation model. The second procedure is to further refine the outputs. This part can be achieved in two ways: one direction is to establish a joint model with the semantic shape constraint. The other technical direction is to make the shape descriptor separated from the joint model and work as an individual unit. In the proposed approach, the minimization problem is transformed from iterative minimization for each variable to automatically minimizing the loss function by learning the generator network parameters. This also leads to a good inductive bias associated with classic variational methods. Extensive experiments have demonstrated the significant advantages.