{"title":"Personalization of industrial human–robot communication through domain adaptation based on user feedback","authors":"","doi":"10.1007/s11257-024-09394-1","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Achieving safe collaboration between humans and robots in an industrial work-cell requires effective communication. This can be achieved through a robot perception system developed using data-driven machine learning. The challenge for human–robot communication is the availability of extensive, labelled datasets for training. Due to the variations in human behaviour and the impact of environmental conditions on the performance of perception models, models trained on standard, publicly available datasets fail to generalize well to domain and application-specific scenarios. Thus, model personalization involving the adaptation of such models to the individual humans involved in the task in the given environment would lead to better model performance. A novel framework is presented that leverages robust modes of communication and gathers feedback from the human partner to auto-label the mode with the sparse dataset. The strength of the contribution lies in using in-commensurable multimodes of inputs for personalizing models with user-specific data. The personalization through feedback-enabled human–robot communication (PF-HRCom) framework is implemented on the use of facial expression recognition as a safety feature to ensure that the human partner is engaged in the collaborative task with the robot. Additionally, PF-HRCom has been applied to a real-time human–robot handover task with a robotic manipulator. The perception module of the manipulator adapts to the user’s facial expressions and personalizes the model using feedback. Having said that, the framework is applicable to other combinations of multimodal inputs in human–robot collaboration applications.</p>","PeriodicalId":49388,"journal":{"name":"User Modeling and User-Adapted Interaction","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"User Modeling and User-Adapted Interaction","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s11257-024-09394-1","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}
引用次数: 0
Abstract
Achieving safe collaboration between humans and robots in an industrial work-cell requires effective communication. This can be achieved through a robot perception system developed using data-driven machine learning. The challenge for human–robot communication is the availability of extensive, labelled datasets for training. Due to the variations in human behaviour and the impact of environmental conditions on the performance of perception models, models trained on standard, publicly available datasets fail to generalize well to domain and application-specific scenarios. Thus, model personalization involving the adaptation of such models to the individual humans involved in the task in the given environment would lead to better model performance. A novel framework is presented that leverages robust modes of communication and gathers feedback from the human partner to auto-label the mode with the sparse dataset. The strength of the contribution lies in using in-commensurable multimodes of inputs for personalizing models with user-specific data. The personalization through feedback-enabled human–robot communication (PF-HRCom) framework is implemented on the use of facial expression recognition as a safety feature to ensure that the human partner is engaged in the collaborative task with the robot. Additionally, PF-HRCom has been applied to a real-time human–robot handover task with a robotic manipulator. The perception module of the manipulator adapts to the user’s facial expressions and personalizes the model using feedback. Having said that, the framework is applicable to other combinations of multimodal inputs in human–robot collaboration applications.
期刊介绍：
User Modeling and User-Adapted Interaction provides an interdisciplinary forum for the dissemination of novel and significant original research results about interactive computer systems that can adapt themselves to their users, and on the design, use, and evaluation of user models for adaptation. The journal publishes high-quality original papers from, e.g., the following areas: acquisition and formal representation of user models; conceptual models and user stereotypes for personalization; student modeling and adaptive learning; models of groups of users; user model driven personalised information discovery and retrieval; recommender systems; adaptive user interfaces and agents; adaptation for accessibility and inclusion; generic user modeling systems and tools; interoperability of user models; personalization in areas such as; affective computing; ubiquitous and mobile computing; language based interactions; multi-modal interactions; virtual and augmented reality; social media and the Web; human-robot interaction; behaviour change interventions; personalized applications in specific domains; privacy, accountability, and security of information for personalization; responsible adaptation: fairness, accountability, explainability, transparency and control; methods for the design and evaluation of user models and adaptive systems
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
