A remote conversation support system for deaf-mute persons based on bimanual gestures recognition using finger-worn devices

2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops) Pub Date : 2015-03-23 DOI:10.1109/PERCOMW.2015.7134101

Kengo Kuroki, Yiming Zhou, Zixue Cheng, Zixian Lu, Yinghui Zhou, Lei Jing

{"title":"A remote conversation support system for deaf-mute persons based on bimanual gestures recognition using finger-worn devices","authors":"Kengo Kuroki, Yiming Zhou, Zixue Cheng, Zixian Lu, Yinghui Zhou, Lei Jing","doi":"10.1109/PERCOMW.2015.7134101","DOIUrl":null,"url":null,"abstract":"Communication between deaf-mute persons and normal people is very difficult when the subjects are out of the objects' view side, or vice verse. This paper presents a hand gesture recognition system using Magic Ring (a ring-shape wearable device) instead of data glove or camera to translate the predefined hand gestures into Japanese and to accomplish remote transmission of the information/command between user groups. The hand gestures are defined based on simplified Japanese sign language. The KNN method is adopted to the pattern recognition according to the bimanual sensing data. To improve the recognition rate, three most similar pre-reserved gestures and similarity are recorded, and Euclidean distance of feature values is used to estimate the gesture similarity. According to the user survey, the system shows good usability and most of the users would like to use such a system. Moreover, in the user-dependent setting, the bimanual accuracy is 85.2%, which is our performance 7.8% by using only right hand.","PeriodicalId":180959,"journal":{"name":"2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2015-03-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"13","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/PERCOMW.2015.7134101","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 13

Abstract

Communication between deaf-mute persons and normal people is very difficult when the subjects are out of the objects' view side, or vice verse. This paper presents a hand gesture recognition system using Magic Ring (a ring-shape wearable device) instead of data glove or camera to translate the predefined hand gestures into Japanese and to accomplish remote transmission of the information/command between user groups. The hand gestures are defined based on simplified Japanese sign language. The KNN method is adopted to the pattern recognition according to the bimanual sensing data. To improve the recognition rate, three most similar pre-reserved gestures and similarity are recorded, and Euclidean distance of feature values is used to estimate the gesture similarity. According to the user survey, the system shows good usability and most of the users would like to use such a system. Moreover, in the user-dependent setting, the bimanual accuracy is 85.2%, which is our performance 7.8% by using only right hand.

查看原文本刊更多论文

一种基于指戴式手势识别的聋哑人远程会话支持系统

聋哑人与正常人之间的交流，当被摄者不在被摄者的视线范围内时，是非常困难的，反之亦然。本文提出了一种手势识别系统，利用Magic Ring(一种环形可穿戴设备)代替数据手套或摄像头，将预定义的手势翻译成日语，实现用户组间信息/命令的远程传递。手势是根据简化的日本手语定义的。根据人工感知数据，采用KNN方法进行模式识别。为了提高识别率，记录三个最相似的预保留手势和相似度，利用特征值的欧氏距离来估计手势的相似度。根据用户调查，该系统显示出良好的可用性，大多数用户愿意使用这样的系统。此外，在依赖于用户的设置中，双手的准确率为85.2%，而我们只使用右手的准确率为7.8%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops)

自引率

0.00%

发文量