Multi-foreground objects segmentation based on RGB-D image

Silhouette extraction of foreground objects appears frequently in various real-world applications, such as Advanced Driving Assistant System, Intelligent Monitoring System, and movie production. Plenty of solutions have been developed to extract silhouette in RGB image with only color information. Since those color based silhouette extraction methods still have difficulties to separate overlapping foreground objects and eliminate excessive segmentation, this paper proposes a novel object segmentation method using color and depth information in RGB-D images. Firstly, we remove the ground plane using the normal map of depth image. Secondly, to separate foreground objects at different distances completely and correctly, the deep Residual Network (ResNets) and Otsu’s multi-thresholding method are combined to divide the depth image into multiple layers. Each depth layer contains only one foreground object or objects at same distance. Finally the outline of foreground object is extracted directly from its depth layer, and refined with color information. Experimental results demonstrate that our method has a better performance than those using color or depth information only, and extracts more types of objects than neural networks.