Segmentation of Shipping Bags in RGB-D Images

Abstract

This paper presents a convolutional neural network (CNN) architecture for segmenting partially transparent shipping bags in RGB-D images of cluttered scenes containing different packaging items in unstructured configurations. The proposed architecture is optimized for training with a limited number of samples with high variability. The analysis of the results with regard to the input type, network architecture, and lighting conditions, proves that including low-resolution depth information improves the segmentation of objects with similar colors and objects in previously unseen lighting conditions, and the high-resolution color photographs greatly improve the segmentation of details. This motivates the proposed multi-input architecture with early feature fusion in order to fully utilize the benefits of high-resolution photographs and low-resolution depth information. The proposed CNN architecture performs successful segmentation of shipping bags in a cluttered environment among packages and items of different colors and materials with irregular shapes. The CNN provides an improvement in accuracy over well-known semantic segmentation architectures while significantly reducing the required processing time, making it suitable for real-time application.