Virtual Reality Intelligent Hardware最新文献

{"title":"A haptic feedback glove for virtual piano interaction","authors":"Yifan FU,&nbsp;Jialin LIU,&nbsp;Xu LI,&nbsp;Xiaoying SUN","doi":"10.1016/j.vrih.2024.07.001","DOIUrl":"10.1016/j.vrih.2024.07.001","url":null,"abstract":"<div><h3>Background</h3><div>Haptic feedback plays a crucial role in virtual reality (VR) interaction, helping to improve the precision of user operation and enhancing the immersion of the user experience. Instrumental haptic feedback in virtual environments is primarily realized using grounded force or vibration feedback devices. However, improvements are required in terms of the active space and feedback realism.</div></div><div><h3>Methods</h3><div>We propose a lightweight and flexible haptic feedback glove that can haptically render objects in VR environments via kinesthetic and vibration feedback, thereby enabling users to enjoy a rich virtual piano-playing experience. The kinesthetic feedback of the glove relies on a cable-pulling mechanism that rotates the mechanism and pulls the two cables connected to it, thereby changing the amount of force generated to simulate the hardness or softness of the object. Vibration feedback is provided by small vibration motors embedded in the bottom of the fingertips of the glove. We designed a piano-playing scenario in the virtual environment and conducted user tests. The evaluation metrics were clarity, realism, enjoyment, and satisfaction.</div></div><div><h3>Results</h3><div>A total of 14 subjects participated in the test, and the results showed that our proposed glove scored significantly higher on the four evaluation metrics than the no-feedback and vibration feedback methods.</div></div><div><h3>Conclusions</h3><div>Our proposed glove significantly enhances the user experience when interacting with virtual objects.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 1","pages":"Pages 95-110"},"PeriodicalIF":0.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143562655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"YGC-SLAM:A visual SLAM based on improved YOLOv5 and geometric constraints for dynamic indoor environments","authors":"Juncheng ZHANG ,&nbsp;Fuyang KE ,&nbsp;Qinqin TANG ,&nbsp;Wenming YU ,&nbsp;Ming ZHANG","doi":"10.1016/j.vrih.2024.05.001","DOIUrl":"10.1016/j.vrih.2024.05.001","url":null,"abstract":"<div><h3>Background</h3><div>As visual simultaneous localization and mapping (SLAM) is primarily based on the assumption of a static scene, the presence of dynamic objects in the frame causes problems such as a deterioration of system robustness and inaccurate position estimation. In this study, we propose a YGC-SLAM for indoor dynamic environments based on the ORB-SLAM2 framework combined with semantic and geometric constraints to improve the positioning accuracy and robustness of the system.</div></div><div><h3>Methods</h3><div>First, the recognition accuracy of YOLOv5 was improved by introducing the convolution block attention model and the improved EIOU loss function, whereby the prediction frame converges quickly for better detection. The improved YOLOv5 was then added to the tracking thread for dynamic target detection to eliminate dynamic points. Subsequently, multi-view geometric constraints were used for re-judging to further eliminate dynamic points while enabling more useful feature points to be retained and preventing the semantic approach from over-eliminating feature points, causing a failure of map building. The K-means clustering algorithm was used to accelerate this process and quickly calculate and determine the motion state of each cluster of pixel points. Finally, a strategy for drawing keyframes with de-redundancy was implemented to construct a clear 3D dense static point-cloud map.</div></div><div><h3>Results</h3><div>Through testing on TUM dataset and a real environment, the experimental results show that our algorithm reduces the absolute trajectory error by 98.22% and the relative trajectory error by 97.98% compared with the original ORB-SLAM2, which is more accurate and has better real-time performance than similar algorithms, such as DynaSLAM and DS-SLAM.</div></div><div><h3>Conclusions</h3><div>The YGC-SLAM proposed in this study can effectively eliminate the adverse effects of dynamic objects, and the system can better complete positioning and map building tasks in complex environments.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"7 1","pages":"Pages 62-82"},"PeriodicalIF":0.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143562653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Survey of neurocognitive disorder detection methods based on speech, visual, and virtual reality technologies","authors":"Tian ZHENG ,&nbsp;Xinheng WANG ,&nbsp;Xiaolan PENG ,&nbsp;Ning SU ,&nbsp;Tianyi XU ,&nbsp;Xurong XIE ,&nbsp;Jin HUANG ,&nbsp;Lun XIE ,&nbsp;Feng TIAN","doi":"10.1016/j.vrih.2024.08.001","DOIUrl":"10.1016/j.vrih.2024.08.001","url":null,"abstract":"<div><div>The global trend of population aging poses significant challenges to society and healthcare systems, particularly because of neurocognitive disorders (NCDs) such as Parkinson's disease (PD) and Alzheimer's disease (AD). In this context, artificial intelligence techniques have demonstrated promising potential for the objective assessment and detection of NCDs. Multimodal contactless screening technologies, such as speech-language processing, computer vision, and virtual reality, offer efficient and convenient methods for disease diagnosis and progression tracking. This paper systematically reviews the specific methods and applications of these technologies in the detection of NCDs using data collection paradigms, feature extraction, and modeling approaches. Additionally, the potential applications and future prospects of these technologies for the detection of cognitive and motor disorders are explored. By providing a comprehensive summary and refinement of the extant theories, methodologies, and applications, this study aims to facilitate an in-depth understanding of these technologies for researchers, both within and outside the field. To the best of our knowledge, this is the first survey to cover the use of speech-language processing, computer vision, and virtual reality technologies for the detection of NSDs.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 6","pages":"Pages 421-472"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143315199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Previs-Real:Interactive virtual previsualization system for news shooting rehearsal and evaluation","authors":"Che Qu ,&nbsp;Shaocong Wang ,&nbsp;Chao Zhou ,&nbsp;Tongchen Zhao ,&nbsp;Rui Guo ,&nbsp;Cheng Wa Wong ,&nbsp;Chi Deng ,&nbsp;Bin Ji ,&nbsp;Yuhui Wen ,&nbsp;Yuanchun Shi ,&nbsp;Yong-Jin Liu","doi":"10.1016/j.vrih.2024.12.001","DOIUrl":"10.1016/j.vrih.2024.12.001","url":null,"abstract":"<div><h3>Background</h3><div>In the demanding field of live news broadcasting, the intricate studio production procedures and tight schedules pose significant challenges for physical rehearsals by cameramen. This paper explores the design and implementation of a lightweight virtual news previsualization system, leveraging virtual production technology and interaction design methods to address the lack of fidelity in presentations and manipulations, and the quantitative feedback of rehearsal effects in previous virtual approaches.</div></div><div><h3>Methods</h3><div>Our system, Previs-Real, is informed by user investigation with professional cameramen and studio technicians, and adheres to principles of high fidelity, accurate replication of actual hardware operations, and real-time feedback on rehearsal results. The system's software and hardware development are implemented based on Unreal Engine and accompanying toolsets, incorporating cutting-edge modeling and camera calibration methods.</div></div><div><h3>Results</h3><div>We validated Previs-Real through a user study, demonstrating superior performance in previsualization shooting tasks using the virtual system compared to traditional camera setups. The findings, supported by both objective performance metrics and subjective responses, underline Previs-Real's effectiveness and potential in transforming news broadcasting rehearsals.</div></div><div><h3>Conclusions</h3><div>Previs-Real eliminates the requirement for complex equipment interconnections and team coordination inherent in a physical studio by implementing methodologies complying the above principles, objectively resulting in a lightweight design of applicable version of virtual news previsualization system. It offers a novel solution to the challenges in news studio previsualization by focusing on key operational features rather than full environment replication. This design approach is equally effective in the process of designing lightweight systems in other fields.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 6","pages":"Pages 527-549"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143315910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MatStick: Changing the material sensation of objects upon impact","authors":"Songxian Liu,&nbsp;Jian He,&nbsp;Shengsheng Jiang,&nbsp;Ziyan Zhang,&nbsp;Mengfei Lv","doi":"10.1016/j.vrih.2024.11.001","DOIUrl":"10.1016/j.vrih.2024.11.001","url":null,"abstract":"<div><div>An increasing number of studies have focused on providing rich tactile feedback in virtual reality interactive scenarios. In this study, we addressed a tapping scenario in virtual reality by designing MatStick, a solution capable of offering diverse tapping sensations. MatStick utilizes a soft physical base to provide force feedback and modulates the instantaneous vibration of the base using a voice coil motor, thereby altering the perception of the base material. We conducted two psychophysical experiments and a subjective evaluation to assess the capabilities of MatStick. The results demonstrate that MatStick can deliver rich tapping sensations. Although users may find it challenging to directly correlate the tapping sensation with the actual physical material based solely on tactile feedback, in immersive scenarios combined with visual and auditory cues, MatStick significantly enhances the user's interaction experience.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 6","pages":"Pages 486-501"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143315200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"InputJump: Augmented reality-facilitated cross-device input fusion based on spatial and semantic information","authors":"Xin Zeng ,&nbsp;Xiaoyu Wang ,&nbsp;Tengxiang Zhang ,&nbsp;Yukang Yan ,&nbsp;Yiqiang Chen","doi":"10.1016/j.vrih.2024.10.001","DOIUrl":"10.1016/j.vrih.2024.10.001","url":null,"abstract":"<div><div>The proliferation of computing devices requires seamless cross-device interactions. Augmented reality (AR) headsets can facilitate interactions with existing computers owing to their user-centered views and natural inputs. In this study, we propose InputJump, a user-centered cross-device input fusion method that maps multi-modal cross-device inputs to interactive elements on graphical interfaces. The input jump calculates the spatial coordinates of the input target positions and the interactive elements within the coordinate system of the AR headset. It also extracts semantic descriptions of inputs and elements using large language models (LLMs). Two types of information from different inputs (e.g., gaze, gesture, mouse, and keyboard) were fused to map onto an interactive element. The proposed method is explained in detail and implemented on both an AR headset and a desktop PC. We then conducted a user study and extensive simulations to validate our proposed method. The results showed that InputJump can accurately associate a fused input with the target interactive element, enabling a more natural and flexible interaction experience.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 6","pages":"Pages 502-526"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143315912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automatic piano performance interaction system based on greedy algorithm for dexterous manipulator","authors":"Yufei Wang ,&nbsp;Junfeng Yao ,&nbsp;Yalan Zhou ,&nbsp;Zefeng Wang","doi":"10.1016/j.vrih.2024.09.001","DOIUrl":"10.1016/j.vrih.2024.09.001","url":null,"abstract":"<div><div>With continuous advancements in artificial intelligence (AI), automatic piano-playing robots have become subjects of cross-disciplinary interest. However, in most studies, these robots served merely as objects of observation with limited user engagement or interaction. To address this issue, we propose a user-friendly and innovative interaction system based on the principles of greedy algorithms. This system features three modules: score management, performance control, and keyboard interactions. Upon importing a custom score or playing a note via an external device, the system performs on a virtual piano in line with user inputs. This system has been successfully integrated into our dexterous manipulator-based piano-playing device, which significantly enhances user interactions.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 6","pages":"Pages 473-485"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143315911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Pre-training transformer with dual-branch context content module for table detection in document images","authors":"Yongzhi Li ,&nbsp;Pengle Zhang ,&nbsp;Meng Sun ,&nbsp;Jin Huang ,&nbsp;Ruhan He","doi":"10.1016/j.vrih.2024.06.003","DOIUrl":"10.1016/j.vrih.2024.06.003","url":null,"abstract":"<div><h3>Background</h3><div>Document images such as statistical reports and scientific journals are widely used in information technology. Accurate detection of table areas in document images is an essential prerequisite for tasks such as information extraction. However, because of the diversity in the shapes and sizes of tables, existing table detection methods adapted from general object detection algorithms, have not yet achieved satisfactory results. Incorrect detection results might lead to the loss of critical information.</div></div><div><h3>Methods</h3><div>Therefore, we propose a novel end-to-end trainable deep network combined with a self-supervised pretraining transformer for feature extraction to minimize incorrect detections. To better deal with table areas of different shapes and sizes, we added a dual-branch context content attention module (DCCAM) to high-dimensional features to extract context content information, thereby enhancing the network's ability to learn shape features. For feature fusion at different scales, we replaced the original 3×3 convolution with a multilayer residual module, which contains enhanced gradient flow information to improve the feature representation and extraction capability.</div></div><div><h3>Results</h3><div>We evaluated our method on public document datasets and compared it with previous methods, which achieved state-of-the-art results in terms of evaluation metrics such as recall and F1-score. <span><span>https://github.com/YongZ-Lee/TD-DCCAM</span><svg><path></path></svg></span></div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 5","pages":"Pages 408-420"},"PeriodicalIF":0.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142587144","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Co-salient object detection with iterative purification and predictive optimization","authors":"Yang Wen,&nbsp;Yuhuan Wang,&nbsp;Hao Wang,&nbsp;Wuzhen Shi,&nbsp;Wenming Cao","doi":"10.1016/j.vrih.2024.06.002","DOIUrl":"10.1016/j.vrih.2024.06.002","url":null,"abstract":"<div><h3>Background</h3><div>Co-salient object detection (Co-SOD) aims to identify and segment commonly salient objects in a set of related images. However, most current Co-SOD methods encounter issues with the inclusion of irrelevant information in the co-representation. These issues hamper their ability to locate co-salient objects and significantly restrict the accuracy of detection.</div></div><div><h3>Methods</h3><div>To address this issue, this study introduces a novel Co-SOD method with iterative purification and predictive optimization (IPPO) comprising a common salient purification module (CSPM), predictive optimizing module (POM), and diminishing mixed enhancement block (DMEB).</div></div><div><h3>Results</h3><div>These components are designed to explore noise-free joint representations, assist the model in enhancing the quality of the final prediction results, and significantly improve the performance of the Co-SOD algorithm. Furthermore, through a comprehensive evaluation of IPPO and state-of-the-art algorithms focusing on the roles of CSPM, POM, and DMEB, our experiments confirmed that these components are pivotal in enhancing the performance of the model, substantiating the significant advancements of our method over existing benchmarks. Experiments on several challenging benchmark co-saliency datasets demonstrate that the proposed IPPO achieves state-of-the-art performance.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 5","pages":"Pages 396-407"},"PeriodicalIF":0.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Music-stylized hierarchical dance synthesis with user control","authors":"Yanbo Cheng,&nbsp;Yichen Jiang,&nbsp;Yingying Wang","doi":"10.1016/j.vrih.2024.06.004","DOIUrl":"10.1016/j.vrih.2024.06.004","url":null,"abstract":"<div><h3>Background</h3><div>Synthesizing dance motions to match musical inputs is a significant challenge in animation research. Compared to functional human motions, such as locomotion, dance motions are creative and artistic, often influenced by music, and can be independent body language expressions. Dance choreography requires motion content to follow a general dance genre, whereas dance performances under musical influence are infused with diverse impromptu motion styles. Considering the high expressiveness and variations in space and time, providing accessible and effective user control for tuning dance motion styles remains an open problem.</div></div><div><h3>Methods</h3><div>In this study, we present a hierarchical framework that decouples the dance synthesis task into independent modules. We use a high-level choreography module built as a Transformer-based sequence model to predict the long-term structure of a dance genre and a low-level realization module that implements dance stylization and synchronization to match the musical input or user preferences. This novel framework allows the individual modules to be trained separately. Because of the decoupling, dance composition can fully utilize existing high-quality dance datasets that do not have musical accompaniments, and the dance implementation can conveniently incorporate user controls and edit motions through a decoder network. Each module is replaceable at runtime, which adds flexibility to the synthesis of dance sequences.</div></div><div><h3>Results</h3><div>Synthesized results demonstrate that our framework generates high-quality diverse dance motions that are well adapted to varying musical conditions and user controls.</div></div>","PeriodicalId":33538,"journal":{"name":"Virtual Reality Intelligent Hardware","volume":"6 5","pages":"Pages 339-357"},"PeriodicalIF":0.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}