Inferring in-air gestures in complex indoor environment with less supervision

IF 3 3区 计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS
Zhongkai Deng , Qizhen Zhou , Jianchun Xing , Qiliang Yang , Yin Chen , Hu Zhang , Zhaoyi Chen , Deyu Deng , Yixin Mo , Bowei Feng
{"title":"Inferring in-air gestures in complex indoor environment with less supervision","authors":"Zhongkai Deng ,&nbsp;Qizhen Zhou ,&nbsp;Jianchun Xing ,&nbsp;Qiliang Yang ,&nbsp;Yin Chen ,&nbsp;Hu Zhang ,&nbsp;Zhaoyi Chen ,&nbsp;Deyu Deng ,&nbsp;Yixin Mo ,&nbsp;Bowei Feng","doi":"10.1016/j.pmcj.2024.101904","DOIUrl":null,"url":null,"abstract":"<div><p>People have high demands for comfort and technology in indoor environments. Gestures, as a natural and friendly human computer interaction (HCI) method, have received widespread attention and have been the subject of many research studies. Traditional approaches are based on wearable devices and cameras, which can be cumbersome to operate and infringe upon users’ privacy. Millimeter-wave (mmWave) radar avoids these problems by detecting gestures in a noninvasive manner. However, it encounters practical challenges in complex indoor environments, such as dynamic disturbance from surroundings, variable usage conditions and diverse gesture patterns, which conventionally require considerable manual effort to address. In this paper, we attempt to minimize human supervision and propose a noninvasive gesture recognition method named RaGe that involves a commercial mmWave indoor radar. First, a parameter optimization framework considering gesture prior constraints is proposed for radar configuration, which functions to weaken the disturbance from surroundings. Second, we alleviate data shortages in variable usage conditions and achieve low-cost data augmentation by applying affine transformations. Third, we combine deformable convolution operations with an unsupervised attention mechanism, thus exploring the intrinsic features involved in diverse gesture patterns. Experimental results show that RaGe is able to recognize 7 gestures with 99.3% accuracy and less human supervision, surpassing the state-of-the-art methods in comparative experiments.</p></div>","PeriodicalId":49005,"journal":{"name":"Pervasive and Mobile Computing","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pervasive and Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1574119224000300","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

People have high demands for comfort and technology in indoor environments. Gestures, as a natural and friendly human computer interaction (HCI) method, have received widespread attention and have been the subject of many research studies. Traditional approaches are based on wearable devices and cameras, which can be cumbersome to operate and infringe upon users’ privacy. Millimeter-wave (mmWave) radar avoids these problems by detecting gestures in a noninvasive manner. However, it encounters practical challenges in complex indoor environments, such as dynamic disturbance from surroundings, variable usage conditions and diverse gesture patterns, which conventionally require considerable manual effort to address. In this paper, we attempt to minimize human supervision and propose a noninvasive gesture recognition method named RaGe that involves a commercial mmWave indoor radar. First, a parameter optimization framework considering gesture prior constraints is proposed for radar configuration, which functions to weaken the disturbance from surroundings. Second, we alleviate data shortages in variable usage conditions and achieve low-cost data augmentation by applying affine transformations. Third, we combine deformable convolution operations with an unsupervised attention mechanism, thus exploring the intrinsic features involved in diverse gesture patterns. Experimental results show that RaGe is able to recognize 7 gestures with 99.3% accuracy and less human supervision, surpassing the state-of-the-art methods in comparative experiments.

在监管较少的情况下推断复杂室内环境中的空中手势
人们对室内环境的舒适度和技术要求很高。手势作为一种自然、友好的人机交互(HCI)方法,受到了广泛关注,并成为许多研究的主题。传统方法以可穿戴设备和摄像头为基础,操作繁琐且侵犯用户隐私。毫米波(mmWave)雷达以非侵入式方式检测手势,从而避免了这些问题。然而,它在复杂的室内环境中遇到了实际挑战,如周围环境的动态干扰、多变的使用条件和多样的手势模式,这些问题通常需要大量的人工操作才能解决。在本文中,我们试图尽量减少人工监督,并提出了一种名为 RaGe 的非侵入式手势识别方法,该方法涉及商用毫米波室内雷达。首先,我们为雷达配置提出了一个考虑到手势先验约束的参数优化框架,其作用是削弱来自周围环境的干扰。其次,我们缓解了多变使用条件下的数据短缺问题,并通过应用仿射变换实现了低成本的数据增强。第三,我们将可变形卷积运算与无监督关注机制相结合,从而探索各种手势模式的内在特征。实验结果表明,RaGe 能够以 99.3% 的准确率识别 7 种手势,且无需人工监督,在对比实验中超越了最先进的方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Pervasive and Mobile Computing
Pervasive and Mobile Computing COMPUTER SCIENCE, INFORMATION SYSTEMS-TELECOMMUNICATIONS
CiteScore
7.70
自引率
2.30%
发文量
80
审稿时长
68 days
期刊介绍: As envisioned by Mark Weiser as early as 1991, pervasive computing systems and services have truly become integral parts of our daily lives. Tremendous developments in a multitude of technologies ranging from personalized and embedded smart devices (e.g., smartphones, sensors, wearables, IoTs, etc.) to ubiquitous connectivity, via a variety of wireless mobile communications and cognitive networking infrastructures, to advanced computing techniques (including edge, fog and cloud) and user-friendly middleware services and platforms have significantly contributed to the unprecedented advances in pervasive and mobile computing. Cutting-edge applications and paradigms have evolved, such as cyber-physical systems and smart environments (e.g., smart city, smart energy, smart transportation, smart healthcare, etc.) that also involve human in the loop through social interactions and participatory and/or mobile crowd sensing, for example. The goal of pervasive computing systems is to improve human experience and quality of life, without explicit awareness of the underlying communications and computing technologies. The Pervasive and Mobile Computing Journal (PMC) is a high-impact, peer-reviewed technical journal that publishes high-quality scientific articles spanning theory and practice, and covering all aspects of pervasive and mobile computing and systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信