采用 MPEG-4 虚拟人物方法的聋哑儿童手语表达系统

3区 计算机科学 Q1 Computer Science
Itimad Raheem Ali, Hoshang Kolivand
{"title":"采用 MPEG-4 虚拟人物方法的聋哑儿童手语表达系统","authors":"Itimad Raheem Ali, Hoshang Kolivand","doi":"10.1007/s12652-024-04842-7","DOIUrl":null,"url":null,"abstract":"<p>Children with language impairments during their significant developmental periods within ‎childhood are exposed to cognitive risk, social impairments, along with language. This is difficult ‎with children born deaf from hearing parents who own little or no experience of communicating ‎in sign language. This system presents the sign language in the context of British ‎Sign Language (BSL) for producing utterances through virtual characters. In capturing, Kinect ‎sensors use a motion capture sensor for motion actors. The connection uses sensors to read data, ‎connect to high-quality 3D scans, and then use these high-quality scans of the animated MPEG-4 ‎face and hand models. The main challenges of this system are the simultaneous capture of data ‎for the whole hand and the development of the MPEG-4 approach considering the animation ‎engines with descriptive sign language features. After synchronizing motion data from motion ‎capture results with Kinect, the combined hand character adjusts points, frames, and time with ‎virtual characters based on the motion of character actors. This study demonstrates the skills of ‎this sign language system instrumental in presenting an assessment by users, highlighting the ‎importance of the hand part in creating new accents and signs in BSL. We have validated this ‎system by testing the reliability and functionality of the virtual characters.‎.</p>","PeriodicalId":14959,"journal":{"name":"Journal of Ambient Intelligence and Humanized Computing","volume":"4 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Expressive sign language system for deaf kids with MPEG-4 approach of virtual human character\",\"authors\":\"Itimad Raheem Ali, Hoshang Kolivand\",\"doi\":\"10.1007/s12652-024-04842-7\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Children with language impairments during their significant developmental periods within ‎childhood are exposed to cognitive risk, social impairments, along with language. This is difficult ‎with children born deaf from hearing parents who own little or no experience of communicating ‎in sign language. This system presents the sign language in the context of British ‎Sign Language (BSL) for producing utterances through virtual characters. In capturing, Kinect ‎sensors use a motion capture sensor for motion actors. The connection uses sensors to read data, ‎connect to high-quality 3D scans, and then use these high-quality scans of the animated MPEG-4 ‎face and hand models. The main challenges of this system are the simultaneous capture of data ‎for the whole hand and the development of the MPEG-4 approach considering the animation ‎engines with descriptive sign language features. After synchronizing motion data from motion ‎capture results with Kinect, the combined hand character adjusts points, frames, and time with ‎virtual characters based on the motion of character actors. This study demonstrates the skills of ‎this sign language system instrumental in presenting an assessment by users, highlighting the ‎importance of the hand part in creating new accents and signs in BSL. We have validated this ‎system by testing the reliability and functionality of the virtual characters.‎.</p>\",\"PeriodicalId\":14959,\"journal\":{\"name\":\"Journal of Ambient Intelligence and Humanized Computing\",\"volume\":\"4 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-14\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Ambient Intelligence and Humanized Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1007/s12652-024-04842-7\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"Computer Science\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Ambient Intelligence and Humanized Computing","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12652-024-04842-7","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Computer Science","Score":null,"Total":0}
引用次数: 0

摘要

有语言障碍的儿童在其童年的重要发展时期面临着认知风险、社交障碍和语言障碍。听力正常的父母很少或根本没有用手语交流的经验,而天生耳聋的儿童则很难做到这一点。该系统在英国手语(BSL)的背景下呈现手语,通过虚拟人物来产生话语。在捕捉过程中,Kinect 传感器使用动作捕捉传感器来捕捉动作演员。连接使用传感器读取数据,连接到高质量三维扫描,然后使用这些高质量扫描的动画 MPEG-4 脸部和手部模型。该系统面临的主要挑战是同步捕捉整只手的数据,以及开发考虑到具有描述性手语特征的动画引擎的 MPEG-4 方法。在将运动捕捉结果中的运动数据与 Kinect 同步后,组合的手部角色会根据角色演员的动作与虚拟角色一起调整点、帧和时间。这项研究展示了这一手语系统在用户评估中的技能,突出了手部在创建新口音和手语符号中的重要性。我们通过测试虚拟人物的可靠性和功能性验证了这一系统。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Expressive sign language system for deaf kids with MPEG-4 approach of virtual human character

Expressive sign language system for deaf kids with MPEG-4 approach of virtual human character

Children with language impairments during their significant developmental periods within ‎childhood are exposed to cognitive risk, social impairments, along with language. This is difficult ‎with children born deaf from hearing parents who own little or no experience of communicating ‎in sign language. This system presents the sign language in the context of British ‎Sign Language (BSL) for producing utterances through virtual characters. In capturing, Kinect ‎sensors use a motion capture sensor for motion actors. The connection uses sensors to read data, ‎connect to high-quality 3D scans, and then use these high-quality scans of the animated MPEG-4 ‎face and hand models. The main challenges of this system are the simultaneous capture of data ‎for the whole hand and the development of the MPEG-4 approach considering the animation ‎engines with descriptive sign language features. After synchronizing motion data from motion ‎capture results with Kinect, the combined hand character adjusts points, frames, and time with ‎virtual characters based on the motion of character actors. This study demonstrates the skills of ‎this sign language system instrumental in presenting an assessment by users, highlighting the ‎importance of the hand part in creating new accents and signs in BSL. We have validated this ‎system by testing the reliability and functionality of the virtual characters.‎.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Ambient Intelligence and Humanized Computing
Journal of Ambient Intelligence and Humanized Computing COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEC-COMPUTER SCIENCE, INFORMATION SYSTEMS
CiteScore
9.60
自引率
0.00%
发文量
854
期刊介绍: The purpose of JAIHC is to provide a high profile, leading edge forum for academics, industrial professionals, educators and policy makers involved in the field to contribute, to disseminate the most innovative researches and developments of all aspects of ambient intelligence and humanized computing, such as intelligent/smart objects, environments/spaces, and systems. The journal discusses various technical, safety, personal, social, physical, political, artistic and economic issues. The research topics covered by the journal are (but not limited to): Pervasive/Ubiquitous Computing and Applications Cognitive wireless sensor network Embedded Systems and Software Mobile Computing and Wireless Communications Next Generation Multimedia Systems Security, Privacy and Trust Service and Semantic Computing Advanced Networking Architectures Dependable, Reliable and Autonomic Computing Embedded Smart Agents Context awareness, social sensing and inference Multi modal interaction design Ergonomics and product prototyping Intelligent and self-organizing transportation networks & services Healthcare Systems Virtual Humans & Virtual Worlds Wearables sensors and actuators
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信