Visual-Tactile Grasp Dataset and Grasp Margin Matrix Analysis for Stability Evaluation

Abstract

Robotic grasping plays a critical role in robotics, with widespread applications across various domains. The stability of a grasp is crucial for subsequent operations, making accurate and robust stability assessments essential. While existing methods predominantly rely on visual data, the absence of tactile signals and quantitative stability metrics often leads to unreliable grasp execution. This article makes three fundamental contributions to address these limitations. First, a visual-tactile grasp dataset generation framework is proposed using NVIDIA Isaac Sim, which synthesizes large-scale multimodal grasping scenarios with stability degree labels through a physics-informed three-stage pipeline. Second, this article introduces the grasp margin matrix, a novel computational model that quantifies directional force margins and rotational moment margins to evaluate grasp robustness. This matrix simplifies traditional grasp wrench space analysis by decoupling complex friction cone calculations into interpretable mechanical metrics, achieving 87.72% stability classification accuracy via our stability assessment network. Third, a vision-based grasp perception network that predicts contact force distributions and object centroids without physical tactile sensors is developed, enabling real-time stability inference through the grasp margin matrix. This perception–evaluation–decision integration links grasp pose generation with stability assessment to inform grasp selection, achieving an 88.15% success rates in real robotic grasping scenarios. Experimental validations demonstrate significant improvements compared to force closure methods, with accuracy improving from 56.52% to 87.72%, offering both theoretical rigor and practical feasibility for industrial robotic systems.