Human–Robot Intrinsic Skill Transfer and Programming by Demonstration System

Robots can often learn skills from human demonstrations. Robot operations via visual perception are commonly influenced by the external environment and are relatively demanding in terms of external conditions, manual segmentation of tasks is time-consuming and labor-intensive, and robots do not perform complex tasks with sufficient accuracy and naturalness in their movements. In this work, we propose a programming by demonstration framework to facilitate autonomous task segmentation and flexible execution. We acquire surface electromyography (sEMG) signals from the forearm and train the gesture datasets by transfer learning from the sign language dataset to achieve action classification. Meanwhile, the inertial information of the forearm is collected and combined with the sEMG signals to autonomously segment operational skills into discrete task units, and after comparing with ground truth, it is demonstrated that multi-modal information can lead to higher segmentation accuracy. To make the robot movements more natural, we add arm stiffness information to this system and estimate the arm stiffness of different individuals by creating a muscle force map of the demonstrator. Finally, human manipulation skills are mapped onto the UR5e robot to validate the results of human-robot skill transfer. Note to Practitioners—The motivation of this work is to record the characteristics of human operations and transfer them to robots for more flexible and safer robot programming, with eventual application in structured industrial scenarios. Differences between our study and others include 1) The use of a portable multi-source signal acquisition device that operates in an uncoupled fashion. 2) Consider the demonstration programming process from the external demonstration information and the inner force information. 3) The results obtained by different people are the same, indicating that the method has good generalization.