Multimodal Technologies and Interaction最新文献_第2页

Active Learning Strategies in Computer Science Education: A Systematic Review 计算机科学教育中的主动学习策略：系统回顾

Multimodal Technologies and Interaction Pub Date : 2024-06-13 DOI: 10.3390/mti8060050

Diana-Margarita Córdova-Esparza, J. Romero-González, Karen-Edith Córdova-Esparza, Juan R. Terven, Rocio-Edith López-Martínez

引用次数: 0

Metaverse & Human Digital Twin: Digital Identity, Biometrics, and Privacy in the Future Virtual Worlds 元宇宙与人类数字双胞胎：未来虚拟世界中的数字身份、生物识别与隐私

Multimodal Technologies and Interaction Pub Date : 2024-06-05 DOI: 10.3390/mti8060048

P. Ruiu, Michele Nitti, Virginia Pilloni, M. Cadoni, Enrico Grosso, M. Fadda

{"title":"Metaverse & Human Digital Twin: Digital Identity, Biometrics, and Privacy in the Future Virtual Worlds","authors":"P. Ruiu, Michele Nitti, Virginia Pilloni, M. Cadoni, Enrico Grosso, M. Fadda","doi":"10.3390/mti8060048","DOIUrl":"https://doi.org/10.3390/mti8060048","url":null,"abstract":"Driven by technological advances in various fields (AI, 5G, VR, IoT, etc.) together with the emergence of digital twins technologies (HDT, HAL, BIM, etc.), the Metaverse has attracted growing attention from scientific and industrial communities. This interest is due to its potential impact on people lives in different sectors such as education or medicine. Specific solutions can also increase inclusiveness of people with disabilities that are an impediment to a fulfilled life. However, security and privacy concerns remain the main obstacles to its development. Particularly, the data involved in the Metaverse can be comprehensive with enough granularity to build a highly detailed digital copy of the real world, including a Human Digital Twin of a person. Existing security countermeasures are largely ineffective and lack adaptability to the specific needs of Metaverse applications. Furthermore, the virtual worlds in a large-scale Metaverse can be highly varied in terms of hardware implementation, communication interfaces, and software, which poses huge interoperability difficulties. This paper aims to analyse the risks and opportunities associated with adopting digital replicas of humans (HDTs) within the Metaverse and the challenges related to managing digital identities in this context. By examining the current technological landscape, we identify several open technological challenges that currently limit the adoption of HDTs and the Metaverse. Additionally, this paper explores a range of promising technologies and methodologies to assess their suitability within the Metaverse context. Finally, two example scenarios are presented in the Medical and Education fields.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"66 s95","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141382753","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring Human Emotions: A Virtual Reality-Based Experimental Approach Integrating Physiological and Facial Analysis 探索人类情感：基于虚拟现实的实验方法，结合生理学和面部分析

Multimodal Technologies and Interaction Pub Date : 2024-06-04 DOI: 10.3390/mti8060047

Leire Bastida, S. Sillaurren, E. Loizaga, Eneko Tomé, Ana Moya

{"title":"Exploring Human Emotions: A Virtual Reality-Based Experimental Approach Integrating Physiological and Facial Analysis","authors":"Leire Bastida, S. Sillaurren, E. Loizaga, Eneko Tomé, Ana Moya","doi":"10.3390/mti8060047","DOIUrl":"https://doi.org/10.3390/mti8060047","url":null,"abstract":"This paper researches the classification of human emotions in a virtual reality (VR) context by analysing psychophysiological signals and facial expressions. Key objectives include exploring emotion categorisation models, identifying critical human signals for assessing emotions, and evaluating the accuracy of these signals in VR environments. A systematic literature review was performed through peer-reviewed articles, forming the basis for our methodologies. The integration of various emotion classifiers employs a ‘late fusion’ technique due to varying accuracies among classifiers. Notably, facial expression analysis faces challenges from VR equipment occluding crucial facial regions like the eyes, which significantly impacts emotion recognition accuracy. A weighted averaging system prioritises the psychophysiological classifier over the facial recognition classifiers due to its higher accuracy. Findings suggest that while combined techniques are promising, they struggle with mixed emotional states as well as with fear and trust emotions. The research underscores the potential and limitations of current technologies, recommending enhanced algorithms for effective interpretation of complex emotional expressions in VR. The study provides a groundwork for future advancements, aiming to refine emotion recognition systems through systematic data collection and algorithm optimisation.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"7 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141267799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Sound of the Police—Virtual Reality Training for Police Communication for High-Stress Operations 警察之声--高压力行动中的警务沟通虚拟现实培训

Multimodal Technologies and Interaction Pub Date : 2024-06-04 DOI: 10.3390/mti8060046

Markus Murtinger, Jakob Carl Uhl, Lisa Maria Atzmüller, Georg Regal, Michael Roither

{"title":"Sound of the Police—Virtual Reality Training for Police Communication for High-Stress Operations","authors":"Markus Murtinger, Jakob Carl Uhl, Lisa Maria Atzmüller, Georg Regal, Michael Roither","doi":"10.3390/mti8060046","DOIUrl":"https://doi.org/10.3390/mti8060046","url":null,"abstract":"Police communication is a field with unique challenges and specific requirements. Police officers depend on effective communication, particularly in high-stress operations, but current training methods are not focused on communication and provide only limited evaluation methods. This work explores the potential of virtual reality (VR) for enhancing police communication training. The rise of VR training, especially in specific application areas like policing, provides benefits. We conducted a field study during police training to assess VR approaches for training communication. The results show that VR is suitable for communication training if factors such as realism, reflection and repetition are given in the VR system. Trainer feedback shows that assistive systems for evaluation and visualization of communication are highly needed. We present ideas and approaches for evaluation in communication training and concepts for visualization and exploration of the data. This research contributes to improving VR police training and has implications for communication training in VR in challenging contexts.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"9 12","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141267347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

User-Centered Evaluation Framework to Support the Interaction Design for Augmented Reality Applications 支持增强现实应用交互设计的用户中心评估框架

Multimodal Technologies and Interaction Pub Date : 2024-05-14 DOI: 10.3390/mti8050041

Andrea Picardi, Giandomenico Caruso

{"title":"User-Centered Evaluation Framework to Support the Interaction Design for Augmented Reality Applications","authors":"Andrea Picardi, Giandomenico Caruso","doi":"10.3390/mti8050041","DOIUrl":"https://doi.org/10.3390/mti8050041","url":null,"abstract":"The advancement of Augmented Reality (AR) technology has been remarkable, enabling the augmentation of user perception with timely information. This progress holds great promise in the field of interaction design. However, the mere advancement of technology is not enough to ensure widespread adoption. The user dimension has been somewhat overlooked in AR research due to a lack of attention to user motivations, needs, usability, and perceived value. The critical aspects of AR technology tend to be overshadowed by the technology itself. To ensure appropriate future assessments, it is necessary to thoroughly examine and categorize all the methods used for AR technology validation. By identifying and classifying these evaluation methods, researchers and practitioners will be better equipped to develop and validate new AR techniques and applications. Therefore, comprehensive and systematic evaluations are critical to the advancement and sustainability of AR technology. This paper presents a theoretical framework derived from a cluster analysis of the most efficient evaluation methods for AR extracted from 399 papers. Evaluation methods were clustered according to the application domains and the human–computer interaction aspects to be investigated. This framework should facilitate rapid development cycles prioritizing user requirements, ultimately leading to groundbreaking interaction methods accessible to a broader audience beyond research and development centers.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"25 11","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140979527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Design and Validation of a Computational Thinking Test for Children in the First Grades of Elementary Education 小学一年级儿童计算思维测试的设计与验证

Multimodal Technologies and Interaction Pub Date : 2024-05-09 DOI: 10.3390/mti8050039

Jorge Hernán Aristizábal Zapata, Julián Esteban Gutiérrez Posada, P. Diago

{"title":"Design and Validation of a Computational Thinking Test for Children in the First Grades of Elementary Education","authors":"Jorge Hernán Aristizábal Zapata, Julián Esteban Gutiérrez Posada, P. Diago","doi":"10.3390/mti8050039","DOIUrl":"https://doi.org/10.3390/mti8050039","url":null,"abstract":"Computational thinking (CT) has garnered significant interest in both computer science and education sciences as it delineates a set of skills that emerge during the problem-solving process. Consequently, numerous assessment instruments aimed at measuring CT have been developed in the recent years. However, a scarce part of the existing CT measurement instruments has been dedicated to early school ages, and few have undergone rigorous validation or reliability testing. Therefore, this work introduces a new instrument for measuring CT in the early grades of elementary education: the Computational Thinking Test for Children (CTTC). To this end, in this work, we provide the design and validation of the CTTC, which is constructed around spatial, sequential, and logical thinking and encompasses abstraction, decomposition, pattern recognition, and coding items organized in five question blocks. The validation and standardization process employs the Kuder–Richardson statistic (KR-20) and expert judgment using V-Aiken for consistency. Additionally, item difficulty indices were utilized to gauge the difficulty level of each question in the CTTC. The study concludes that the CTTC demonstrates consistency and suitability for children in the first cycle of primary education (encompassing the first to third grades).","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":" 16","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140996422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multimodal Embodiment Research of Oral Music Traditions: Electromyography in Oud Performance and Education Research of Persian Art Music 口头音乐传统的多模态体现研究：乌德琴演奏中的肌电图和波斯艺术音乐教育研究

Multimodal Technologies and Interaction Pub Date : 2024-05-07 DOI: 10.3390/mti8050037

Stella Paschalidou

{"title":"Multimodal Embodiment Research of Oral Music Traditions: Electromyography in Oud Performance and Education Research of Persian Art Music","authors":"Stella Paschalidou","doi":"10.3390/mti8050037","DOIUrl":"https://doi.org/10.3390/mti8050037","url":null,"abstract":"With the recent advent of research focusing on the body’s significance in music, the integration of physiological sensors in the context of empirical methodologies for music has also gained momentum. Given the recognition of covert muscular activity as a strong indicator of musical intentionality and the previously ascertained link between physical effort and various musical aspects, electromyography (EMG)—signals representing muscle activity—has also experienced a noticeable surge. While EMG technologies appear to hold good promise for sensing, capturing, and interpreting the dynamic properties of movement in music, which are considered innately linked to artistic expressive power, they also come with certain challenges, misconceptions, and predispositions. The paper engages in a critical examination regarding the utilisation of muscle force values from EMG sensors as indicators of physical effort and musical activity, particularly focusing on (the intuitively expected link to) sound levels. For this, it resides upon empirical work, namely practical insights drawn from a case study of music performance (Persian instrumental music) in the context of a music class. The findings indicate that muscle force can be explained by a small set of (six) statistically significant acoustic and movement features, the latter captured by a state-of-the-art (full-body inertial) motion capture system. However, no straightforward link to sound levels is evident.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"7 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141004460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Narrative Review of the Sociotechnical Landscape and Potential of Computer-Assisted Dynamic Assessment for Children with Communication Support Needs 针对有交流支持需求儿童的计算机辅助动态评估的社会技术前景和潜力的叙述性回顾

Multimodal Technologies and Interaction Pub Date : 2024-05-07 DOI: 10.3390/mti8050038

Christopher S. Norrie, Stijn R. J. M. Deckers, Maartje Radstaake, Hans van Balkom

{"title":"A Narrative Review of the Sociotechnical Landscape and Potential of Computer-Assisted Dynamic Assessment for Children with Communication Support Needs","authors":"Christopher S. Norrie, Stijn R. J. M. Deckers, Maartje Radstaake, Hans van Balkom","doi":"10.3390/mti8050038","DOIUrl":"https://doi.org/10.3390/mti8050038","url":null,"abstract":"This paper presents a narrative review of the current practices in assessing learners’ cognitive abilities and the limitations of traditional intelligence tests in capturing a comprehensive understanding of a child’s learning potential. Referencing prior research, it explores the concept of dynamic assessment (DA) as a promising yet underutilised alternative that focuses on a child’s responsiveness to learning opportunities. The paper highlights the potential of novel technologies, in particular tangible user interfaces (TUIs), in integrating computational science with DA to improve the access and accuracy of assessment results, especially for children with communication support needs (CSN), as a catalyst for abetting critical communicative competencies. However, existing research in this area has mainly focused on the automated mediation of DA, neglecting the human element that is crucial for effective solutions in special education. A framework is proposed to address these issues, combining pedagogical and sociocultural elements alongside adaptive information technology solutions in an assessment system informed by user-centred design principles to fully support teachers/facilitators and learners with CSN within the special education ecosystem.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"13 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141004868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Saliency-Guided Point Cloud Compression for 3D Live Reconstruction 显著性引导的点云压缩用于三维实时重建

Multimodal Technologies and Interaction Pub Date : 2024-05-03 DOI: 10.3390/mti8050036

P. Ruiu, Lorenzo Mascia, Enrico Grosso

{"title":"Saliency-Guided Point Cloud Compression for 3D Live Reconstruction","authors":"P. Ruiu, Lorenzo Mascia, Enrico Grosso","doi":"10.3390/mti8050036","DOIUrl":"https://doi.org/10.3390/mti8050036","url":null,"abstract":"3D modeling and reconstruction are critical to creating immersive XR experiences, providing realistic virtual environments, objects, and interactions that increase user engagement and enable new forms of content manipulation. Today, 3D data can be easily captured using off-the-shelf, specialized headsets; very often, these tools provide real-time, albeit low-resolution, integration of continuously captured depth maps. This approach is generally suitable for basic AR and MR applications, where users can easily direct their attention to points of interest and benefit from a fully user-centric perspective. However, it proves to be less effective in more complex scenarios such as multi-user telepresence or telerobotics, where real-time transmission of local surroundings to remote users is essential. Two primary questions emerge: (i) what strategies are available for achieving real-time 3D reconstruction in such systems? and (ii) how can the effectiveness of real-time 3D reconstruction methods be assessed? This paper explores various approaches to the challenge of live 3D reconstruction from typical point cloud data. It first introduces some common data flow patterns that characterize virtual reality applications and shows that achieving high-speed data transmission and efficient data compression is critical to maintaining visual continuity and ensuring a satisfactory user experience. The paper thus introduces the concept of saliency-driven compression/reconstruction and compares it with alternative state-of-the-art approaches.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":"104 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141016212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Into the Rhythm: Evaluating Breathing Instruction Sound Experiences on the Run with Novice Female Runners 融入节奏：评估新手女选手跑步时的呼吸指导声音体验

Multimodal Technologies and Interaction Pub Date : 2024-03-22 DOI: 10.3390/mti8040025

Vincent van Rheden, Eric Harbour, Thomas Finkenzeller, Alexander Meschtscherjakov

{"title":"Into the Rhythm: Evaluating Breathing Instruction Sound Experiences on the Run with Novice Female Runners","authors":"Vincent van Rheden, Eric Harbour, Thomas Finkenzeller, Alexander Meschtscherjakov","doi":"10.3390/mti8040025","DOIUrl":"https://doi.org/10.3390/mti8040025","url":null,"abstract":"Running is a popular sport throughout the world. Breathing strategies like stable breathing and slow breathing can positively influence the runner’s physiological and psychological experiences. Sonic breathing instructions are an established, unobtrusive method used in contexts such as exercise and meditation. We argue sound to be a viable approach for administering breathing strategies whilst running. This paper describes two laboratory studies using within-subject designs that investigated the usage of sonic breathing instructions with novice female runners. The first study (N = 11) examined the effect of information richness of five different breathing instruction sounds on adherence and user experience. The second study (N = 11) explored adherence and user experience of sonically more enriched sounds, and aimed to increase the sonic experience. Results showed that all sounds were effective in stabilizing the breathing rate (study 1 and 2, respectively: mean absolute percentage error = 1.16 ± 1.05% and 1.9 ± 0.11%, percent time attached = 86.81 ± 9.71% and 86.18 ± 11.96%). Information-rich sounds were subjectively more effective compared to information-poor sounds (mean ratings: 7.55 ± 1.86 and 5.36 ± 2.42, respectively). All sounds scored low (mean < 5/10) on intention to use.","PeriodicalId":508555,"journal":{"name":"Multimodal Technologies and Interaction","volume":" 32","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140215829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0