Box segmentation, position and size estimation for robotic box handling applications

Abstract

This work presents a vision system developed for the robotic handling of boxes in logistic applications. The fundamental problem is to reliably and accurately estimate the position and size of boxes in the environment to enable successful and safe handling. To sense the environment we use color and depth images coming from RGB and Time-of-Flight cameras. The vision task is divided into two sub-problems. First, an instance segmentation problem in the color image space approached using deep learning to detect objects; second, a position, orientation and size estimation problem is solved using geometric algorithms over the input point cloud of each object. Our proposed approach achieves an average position estimation error of 7mm and size estimation error of 8mm. We demonstrate the effectiveness of our approach in real tests by handling carton and Styrofoam boxes of various sizes with a 6 Degree of Freedom robot using our proposed vision system.