{"title":"Biomimetic learning of hand gestures in a humanoid robot","authors":"Parthan Olikkal, Dingyi Pei, Bharat Kashyap Karri, Ashwin Satyanarayana, Nayan M. Kakoty, Ramana Vinjamuri","doi":"10.3389/fnhum.2024.1391531","DOIUrl":null,"url":null,"abstract":"Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.","PeriodicalId":12536,"journal":{"name":"Frontiers in Human Neuroscience","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers in Human Neuroscience","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3389/fnhum.2024.1391531","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"NEUROSCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.
期刊介绍:
Frontiers in Human Neuroscience is a first-tier electronic journal devoted to understanding the brain mechanisms supporting cognitive and social behavior in humans, and how these mechanisms might be altered in disease states. The last 25 years have seen an explosive growth in both the methods and the theoretical constructs available to study the human brain. Advances in electrophysiological, neuroimaging, neuropsychological, psychophysical, neuropharmacological and computational approaches have provided key insights into the mechanisms of a broad range of human behaviors in both health and disease. Work in human neuroscience ranges from the cognitive domain, including areas such as memory, attention, language and perception to the social domain, with this last subject addressing topics, such as interpersonal interactions, social discourse and emotional regulation. How these processes unfold during development, mature in adulthood and often decline in aging, and how they are altered in a host of developmental, neurological and psychiatric disorders, has become increasingly amenable to human neuroscience research approaches. Work in human neuroscience has influenced many areas of inquiry ranging from social and cognitive psychology to economics, law and public policy. Accordingly, our journal will provide a forum for human research spanning all areas of human cognitive, social, developmental and translational neuroscience using any research approach.