Effects of Force-Torque and Tactile Haptic Modalities on Classifying the Success of Robot Manipulation Tasks

Abstract

We investigate which haptic sensing modalities, or combination of haptic sensing modalities, best enable a robot to determine whether it successfully completed a manipulation task. In this paper, we consider haptic sensing modalities obtained from a wrist-mounted force-torque sensor and three types of fingertip sensors: a pair of FlexiForce force-sensing resistors, a pair of NumaTac sensors, and a pair of BioTac sensors. For each type of fingertip sensor, we simultaneously record force-torque and fingertip tactile data as the robot attempted to complete two manipulation tasks—a picking task and a scooping task—two-hundred times each. We leverage the resulting dataset to train and test a classification method using forty-one different haptic feature combinations, obtained from exhaustive combinations of individual modalities of the force-torque sensor and fingertip sensors. Our results show that the classification method’s ability to distinguish between successful and unsuccessful task attempts depends on both the type of manipulation task and the subset of haptic modalities used to train and test the classification method.