ViTract: Robust Object Shape Perception via Active Visuo-Tactile Interaction

An essential problem in robotic systems that are to be deployed in unstructured environments is the accurate and autonomous perception of the shapes of previously unseen objects. Existing methods for shape estimation or reconstruction have leveraged either visual or tactile interactive exploration techniques or have relied on comprehensive visual or tactile information acquired in an offline manner. In this letter, a novel visuo-tactile interactive perception framework- ViTract, is introduced for shape estimation of unseen objects. Our framework estimates the shape of diverse objects robustly using low-dimensional, efficient, and generalizable shape primitives, which are superquadrics. The probabilistic formulation within our framework takes advantage of the complementary information provided by vision and tactile observations while accounting for associated noise. As part of our framework, we propose a novel modality-specific information gain to select the most informative and reliable exploratory action (using vision/tactile) to obtain iterative visuo/tactile information. Our real-robot experiments demonstrate superior and robust performance compared to state-of-the-art visuo-tactile-based shape estimation techniques.