Grasping Force Estimation for Markerless Visuotactile Sensors

Tactile sensors have been used for force estimation in the past, especially vision-based tactile sensors (VBTSs) have recently become a new trend due to their high spatial resolution and low cost. In this work, we have designed and implemented several approaches to estimate the normal grasping force using different types of markerless visuotactile representations obtained from VBTS. Our main goal is to determine the most appropriate visuotactile representation, based on a performance analysis during robotic grasping tasks. Our proposal has been tested on the dataset generated with our DIGIT sensors and another one obtained using GelSight Mini sensors from another state-of-the-art work. We have also tested the generalization capabilities of our best approach, called RGBmod. The results led to two main conclusions. First, the RGB visuotactile representation is a better input option than the depth image or a combination of the two for estimating normal grasping forces. Second, RGBmod achieved a good performance when tested on ten unseen everyday objects in real-world scenarios, achieving an average relative error (RE) of $0.125~\pm ~0.153$ . Furthermore, we show that our proposal outperforms other works in the literature that use RGB and depth information for the same task.