Measuring Bending Angle and Hallucinating Shape of Elongated Deformable Objects

Abstract

Many objects in a human-made environment have elongated shapes for easy manipulation and grasping. As humanoid robots are working in this environment, they require proper sensing and perception of such objects. Current approaches are providing mainly the perception of rigid objects, but many everyday items are non-rigid and more challenging to track due to their substantial shape variability. We want the robots to be able to grasp and manipulate thin, elongated, deformable objects. We propose a system based on the Deep Neural Network that can predict the bend angle of such objects using the single RGB image only. In our paper, we present the proposed neural network architecture used for prediction of the bending angle and finding the elongated shape in images with a cluttered background together with the dataset used for training. We observed that the proposed system even though it was trained on synthetic data was able to perform well on real data. The proposed architecture also provide us with the ability to hallucinate how the deformable pipe with any initial bend would look like when subjected to the arbitrary bend angle. Our findings have more profound consequences than the above mentioned. We were able to show that the proposed Encoder-Decoder neural network architecture has the interpretable latent vector element for describing a measurable physical bend angle. Moreover, we allow bending arrows to be situated out of the image plane. In the future work, we are planning to extend the current approach with the prediction of the full 3d shape of the elongated object from a single RGB image.