Soft objects grasping evaluation using a novel VCFN-YOLOv8 framework

Humans can quickly perform adaptive grasping of soft objects by using visual perception and judgment of the grasping angle, which helps prevent the objects from sliding or deforming excessively. However, this easy task remains a challenge for robots. The grasping states of soft objects can be categorized into four types: sliding, appropriate, excessive and extreme. Effective recognition of different states is crucial for achieving adaptive grasping of soft objects. To address this problem, a novel visual-curvature fusion network based on YOLOv8 (VCFN-YOLOv8) is proposed to evaluate the grasping state of various soft objects. In this framework, the robotic arm equipped with the wrist camera and the curvature sensor is established to perform generalization grasping and lifting experiments on 11 different objects. Meanwhile, the dataset is built for training and testing the proposed method. The results show a classification accuracy of 99.51% on four different grasping states. A series of grasping evaluation experiments is conducted based on the proposed framework, along with tests for the model’s generality. The experiment results demonstrate that VCFN-YOLOv8 is accurate and efficient in evaluating the grasping state of soft objects and shows a certain degree of generalization for non-soft objects. It can be widely applied in fields such as automatic control, adaptive grasping and surgical robot.