2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)最新文献

CuriosityXR: Context-aware Education Experiences with Mixed Reality and Conversation AI CuriosityXR：利用混合现实和人工智能对话实现情境感知教育体验

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00013

Aaditya Vaze, Alexis Morris, Ian Clarke

{"title":"CuriosityXR: Context-aware Education Experiences with Mixed Reality and Conversation AI","authors":"Aaditya Vaze, Alexis Morris, Ian Clarke","doi":"10.1109/AIxVR59861.2024.00013","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00013","url":null,"abstract":"The educational landscape is undergoing a fundamental shift towards a learner-centric model, emphasizing engagement, interaction, and personalization in the learning process. This study investigates new technologies that enable immersive, self-guided, and curiosity-driven educational experiences, addressing these crucial elements. The research delves into Mixed Reality (MR) as a tool for constructing a context-aware system that nurtures learners’ inquisitiveness while enhancing memory retention. The paper presents the design and development of \"Curiosity XR,\" an MR headset application created using a research-through-design methodology, acting as a platform for educators to develop contextual and multi-modal interactive mini-lessons. Learners can engage with these lessons and also benefit from AI-supported learning content. The evaluation of this design involves a user participant study and subsequent interviews, revealing greater engagement levels, increased curiosity to learn, and improved visual content retention among participants. This work aims to encourage further exploration within the MR domain and promote the integration of MR and AI for the advancement of curiosity-driven education.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"16 5","pages":"41-49"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Instant Difficulty Adjustment using User Skill Model Based on GPDM in VR Kendama Task 在 VR Kendama 任务中使用基于 GPDM 的用户技能模型即时调整难度

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00026

Yusuke Goutsu, Tetsunari Inamura

{"title":"Instant Difficulty Adjustment using User Skill Model Based on GPDM in VR Kendama Task","authors":"Yusuke Goutsu, Tetsunari Inamura","doi":"10.1109/AIxVR59861.2024.00026","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00026","url":null,"abstract":"Adapting to user’s skill is crucial to task difficulty adjustment. This paper presents a task difficulty adjustment method that predicts future success rate when changing the difficulty level with small data from each user: instant difficulty adjustment. We proposed a methodology based on a Gaussian process dynamical model (GPDM) to model the user’s skill from past performance observations, and predict future performance at a targeted difficulty level stochastically. As a task to be performed, we focused on a cup-and-ball game (a kind of juggling called Kendama) using virtual reality (VR), in which the cup size is changeable to adjust the difficulty level in a VR environment. In the experiment, we compared the proposed method with LSTM-based deterministic method set by randomized initial parameters with participants who had different skills of the Kendama task. Our results indicate that the GPDM-based method accurately reflects the user’s skill, and the predicted success rate is close to the actual success rate, which tends to appear under the conditions of balanced training data on the number of successes and failures. Additionally, our method is valid for decreasing the training data, which means the prediction accuracy is ensured even with a small number of Kendama trials. In future work, we will achieve the instant difficulty adjustment at various training data not restricted to the number of successes and failures.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"81 5","pages":"138-146"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

VRmonic: A VR Piano Playing Form Trainer VRmonic：VR 钢琴弹奏形式训练器

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00056

Ethan Matzek, Tyler Yankee, Odin Kohler, Tracy Lipke-Perry, N. Banerjee, Sean Banerjee

{"title":"VRmonic: A VR Piano Playing Form Trainer","authors":"Ethan Matzek, Tyler Yankee, Odin Kohler, Tracy Lipke-Perry, N. Banerjee, Sean Banerjee","doi":"10.1109/AIxVR59861.2024.00056","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00056","url":null,"abstract":"Improper hand form during practice and performance can lead to long-term injuries in musicians of all levels. We present a demonstration of VRmonic, a novel immersive VR-based piano trainer for learning correct hand form for piano by overlaying the playing form of an expert oracle. VRmonic consists of a library of 48 scales comprising of the major, and natural, harmonic, and melodic minor scales for each of the 12 notes. We obtain correct hand form by using an automatic approach to detect hand keypoints from recordings of an expert pianist collected using an overhead RGB-D mounted camera. During training, the user can use VRmonic to load in a prior recording of themselves playing a scale and compare their hand form against the expert oracle. During playback, we display virtual oracle and user hand overlap via a color change, controlled by our form tolerance threshold slider that allows the user to determine how aggressively VRmonic should check incorrect form. VRmonic enables everyday users to practice piano playing with continuous feedback while mitigating long-term physical injury.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"41 7-8","pages":"330-334"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531682","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Equipment Assembly Recognition for Augmented Reality Guidance 用于增强现实制导的设备装配识别系统

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00023

Kevin Murray, Jonathan Schierl, Kevin Foley, Zoran Duric

{"title":"Equipment Assembly Recognition for Augmented Reality Guidance","authors":"Kevin Murray, Jonathan Schierl, Kevin Foley, Zoran Duric","doi":"10.1109/AIxVR59861.2024.00023","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00023","url":null,"abstract":"Equipment maintenance is a challenging task, particularly for complex equipment with many parts. Augmented Reality (AR) technology can assist technicians by providing real-time, on-site guidance. A fundamental requirement for this guidance is recognizing the current pose and assembly state of the equipment. This work addresses the problem of recognizing the pose and assembly state of equipment from multiple visible light images, specifically aiming to handle real-world equipment with hundreds of parts and millions of possible assembly states. We propose a novel two-stage method that first estimates a coarse pose and assembly state, then refines these estimates by leveraging multi-view integration of 2D features in a 3D voxel grid. Our approach is validated on two real assemblies with hundreds of parts: a small engine and a 3D printer. Experimental results demonstrate the effectiveness of our method, with refinement improving both pose and assembly state estimates. This work contributes a new perspective to AR-guided equipment maintenance, highlighting the importance of valid assembly states in training and the benefits of multi-view feature integration for assembly recognition.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"138 1","pages":"109-118"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531465","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards Situated Imaging 实现情景成像

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00019

Mingze Xi, Madhawa Perera, Stuart Anderson, Matt Adcock

{"title":"Towards Situated Imaging","authors":"Mingze Xi, Madhawa Perera, Stuart Anderson, Matt Adcock","doi":"10.1109/AIxVR59861.2024.00019","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00019","url":null,"abstract":"Integrating augmented reality (AR) with externally hosted computer vision (CV) models can provide enhanced AR experiences. For instance, by utilising an advanced object detection model, an AR system can recognise a range of predefined objects within the user’s immediate surroundings. However, existing AR-CV workflows rarely incorporate user-defined contextual information, which often come in the form of multi-modal queries blending both natural and body language. Interpreting these intricate user queries, processing them via a sequence of deep learning models, and then adeptly visualising the outcomes remains a formidable challenge.In this paper, we describe Situated Imaging (SI), an extensible array of techniques for in-situ interactive visual computing. We delineate the architecture of the Situated Imaging framework, which enhances the conventional AR-CV workflow by incorporating a range of advanced interactive and generative computer vision techniques. We also describe a demonstration implementation to illustrate the pipeline’s capabilities, enabling users to engage in activities such as labelling, highlighting, or generating content within a user-defined context. Furthermore, we provide initial guidance for tailoring this framework to example use cases and identify avenues for future research. Our model-agnostic Situated Imaging pipeline acts as a valuable starting point for both academic scholars and industry practitioners interested in enhancing the AR experience by incorporating computationally intensive AI models.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"67 3","pages":"85-89"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Interactive Digital Twins for Online Planning and Worker Safety in Intralogistics and Production 用于内部物流和生产中在线规划和工人安全的交互式数字双胞胎

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00016

Andreas Luxenburger, Jonas Mohr, Dieter Merkel, Sönke Knoch, Daniel Porta, Christian Paul, Johannes Widenka, Philipp Schäfers, Markus Baumann, Stefan Lehnhoff, Joachim Schwab

{"title":"Interactive Digital Twins for Online Planning and Worker Safety in Intralogistics and Production","authors":"Andreas Luxenburger, Jonas Mohr, Dieter Merkel, Sönke Knoch, Daniel Porta, Christian Paul, Johannes Widenka, Philipp Schäfers, Markus Baumann, Stefan Lehnhoff, Joachim Schwab","doi":"10.1109/AIxVR59861.2024.00016","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00016","url":null,"abstract":"Asset administration shells constitute a standardized, technical realization of digital twins and enable a unified description of important information within a production site for a safe, continuous and joint planning of intralogistics and manufacturing processes. In this paper, we show how an information and service infrastructure based on asset administration shells can be used to provide online-planners with unified, i.e. manufacturer-independent information facets of and respective control interfaces for involved assets, such as collaborative robots. To this end, we implemented a semi-virtualized production process using important industrial communication standards where the current state of real and simulated robots, as well as products, can be observed and manipulated at run-time and on site by means of an Augmented Reality-based user interface. In such dynamic scenarios, the worker safety needs to be maintained using the advances from the virtual representation of all assets. A technical demonstrator has been developed and discussed with domain experts to prove the underlying concept.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"212 5","pages":"66-74"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

P5G: A Patient-Centered Design Method of Virtual Reality Health Game System for Children's Amblyopia Rehabilitation P5G：以患者为中心的儿童弱视康复虚拟现实健康游戏系统设计方法

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00041

Ye Yan, Xiaomei Nie, Yuhan Dong

{"title":"P5G: A Patient-Centered Design Method of Virtual Reality Health Game System for Children's Amblyopia Rehabilitation","authors":"Ye Yan, Xiaomei Nie, Yuhan Dong","doi":"10.1109/AIxVR59861.2024.00041","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00041","url":null,"abstract":"Amblyopia is a prevalent ophthalmic disease that occurs during the sensitive period of children's visual development, which can have adverse effects on patients' visual and psychological well-being. In recent years, there has been a significant focus on digital amblyopia therapy, particularly with advancements in virtual reality (VR) technology and functional game design. This study is based on the P5 eHealth approach, a theory of digital healthcare that identifies five essential properties of a user-centered eHealth system, including preventive, predictive, personalized, participatory, and psycho-cognitive. In this paper, we present a P5G design framework, which refers to P5 eHealth for health games, and we have utilized this framework for personalized health game design. Additionally, we have developed a VR game system for children's amblyopia rehabilitation to demonstrate the effectiveness of the P5G design framework. The proposed system incorporates visual function principles and amblyopia rehabilitation strategies, prioritizes binocular vision, and consists of multiple modules for visual training.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"69 3","pages":"251-255"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Holo-Museum: An Interactive Holographic Display System Based on 3D Scanning by a LiDAR for a Solid Art Sculpture 全息博物馆：基于激光雷达对实体艺术雕塑进行 3D 扫描的交互式全息显示系统

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00068

Mei-Yuan Zeng, Shih-Wei Sun

引用次数: 0

Towards Efficient Visual Attention Prediction for 360 Degree Videos 实现 360 度视频的高效视觉注意力预测

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00014

Herman Prawiro, Tse-Yu Pan, Chun-Kai Yang, Chih-Tsun Huang, Min-Chun Hu

引用次数: 0

TransUser's: A Transformer Based Salient Object Detection for Users Experience Generation in 360° Videos TransUser's：基于变换器的突出物体检测，用于生成 360° 视频中的用户体验

2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) Pub Date : 2024-01-17 DOI: 10.1109/AIxVR59861.2024.00042

I. Khan, Kyungjin Han, Jong Weon Lee

{"title":"TransUser's: A Transformer Based Salient Object Detection for Users Experience Generation in 360° Videos","authors":"I. Khan, Kyungjin Han, Jong Weon Lee","doi":"10.1109/AIxVR59861.2024.00042","DOIUrl":"https://doi.org/10.1109/AIxVR59861.2024.00042","url":null,"abstract":"A 360-degree video stream enables users to view their point of interest while giving them the sense of 'being there'. Performing head or hand manipulations to watch the salient objects and sceneries in such a video is a very tiresome task and the user may miss the interesting events. Compared to this, the automatic selection of a user's Point of Interest (PoI) in a 360° video is extremely challenging due to subjective viewpoints and varying degrees of satisfaction. To handle these challenges, we employed an attention-based transformer approach to detect salient objects inside the immersive contents. In the proposed framework, first, an input 360° video is converted into frames where each frame is passed to a CNNbased encoder. The CNN encoder generates feature maps of the input framers. Further, for an attention-based network, we used a stack of three transformers encoder with position embeddings to generate position-awareness embeddings of the encoded feature maps. Each transformer encoder is based on a multihead self-attention block and a multi-layer perceptron with various sets of attention blocks. Finally, encoded features and position embeddings from the transformer encoder are passed through a CNN decoder network to predict the salient object inside the 360° video frames. We evaluated our results on four immersive videos to find the effectiveness of the proposed framework. Further, we also compared our results with state-of-the-art methods where the proposed method outperformed the other existing models.","PeriodicalId":518749,"journal":{"name":"2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)","volume":"194 2","pages":"256-260"},"PeriodicalIF":0.0,"publicationDate":"2024-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140531142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0