Initiating GrabCut by color difference for automatic foreground extraction of passport imagery

Abstract

Grabcut, an iterative algorithm based on Graph Cut, is a popular foreground segmentation method. However, it suffers from a main drawback: a manual interaction is required in order to start segmenting the image. In this paper, four different methods based on image pairs are used to obtain an initial extraction of the foreground. Then, the obtained initial estimation of the foreground is used as input to the GrabCut algorithm, thus avoiding the need of interaction. Moreover, this paper is focused on passport images, which require an almost pixel-perfect segmentation in order to be a valid photo. Having gathered our own dataset and generated ground truth images, promising results are obtained in terms of F1-scores, with a maximum mean of 0.975 among all the images, improving the performance of GrabCut in all cases. Some future work directions are given for those unsolved issues that were faced, such as the segmentation in hair regions or tests in a non-uniform background scenario.