Visual sensing approach for high-precision pressure estimation in robotic manipulation

Abstract

Accurate sensing of contact pressure in robotic grippers is essential for both autonomous and teleoperated manipulation. However, the direct integration of tactile sensors onto gripper surfaces often poses significant challenges. With the advancement of high-resolution cameras and computer vision technologies, vision-based pressure estimation methods have emerged as viable alternatives. This paper presents a novel high-precision vision-based pressure estimation (HiVPE) method that addresses limitations in existing approaches, particularly when dealing with posture changes, low image quality, environmental variations, and occlusion. The key contributions of this work are in the realm of artificial intelligence (AI), where a novel dual-channel feature enhancement architecture is proposed, comprising a Spatial Feature Modulation (SFM) module for detailed manipulator-specific perception and a Visual Pressure Mamba (VP-Mamba) module based on state space models for capturing long-range dependencies and global structural information. In the engineering domain, the application of this AI technology involves implementing an eye-in-hand camera system to generate pressure maps from gripper images, enabling robust pressure estimation across diverse real-world scenarios without direct sensor integration. Experimental evaluations in both controlled laboratory settings and diverse real-world environments demonstrate excellent performance, with a temporal accuracy of 96.22% and 95.48% for tendon-actuated and soft grippers, respectively, while reducing the mean absolute error by 5.6% and 19.5% compared to baseline methods. These results validate the superior performance of HiVPE in complex operational tasks and unknown environments, making it a feasible solution for practical robotic applications.