Multimodal Technol. Interact.最新文献

{"title":"Federated Learning for Clinical Event Classification Using Vital Signs Data","authors":"Ruzaliev Rakhmiddin, Kangyoon Lee","doi":"10.3390/mti7070067","DOIUrl":"https://doi.org/10.3390/mti7070067","url":null,"abstract":"Accurate and timely diagnosis is a pillar of effective healthcare. However, the challenge lies in gathering extensive training data while maintaining patient privacy. This study introduces a novel approach using federated learning (FL) and a cross-device multimodal model for clinical event classification based on vital signs data. Our architecture employs FL to train several machine learning models including random forest, AdaBoost, and SGD ensemble models on vital signs data. The data were sourced from a diverse clientele at a Boston hospital (MIMIC-IV dataset). The FL structure trains directly on each client’s device, ensuring no transfer of sensitive data and preserving patient privacy. The study demonstrates that FL offers a powerful tool for privacy-preserving clinical event classification, with our approach achieving an impressive accuracy of 98.9%. These findings highlight the significant potential of FL and cross-device ensemble technology in healthcare applications, especially in the context of handling large volumes of sensitive patient data.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130546748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Dynamic Interactive Approach to Music Listening: The Role of Entrainment, Attunement and Resonance","authors":"M. Reybrouck","doi":"10.3390/mti7070066","DOIUrl":"https://doi.org/10.3390/mti7070066","url":null,"abstract":"This paper takes a dynamic interactive stance to music listening. It revolves around the focal concept of entrainment as an operational tool for the description of fine-grained dynamics between the music as an entraining stimulus and the listener as an entrained subject. Listeners, in this view, can be “entrained” by the sounds at several levels of processing, dependent on the degree of attunement and alignment of their attention. The concept of entrainment, however, is somewhat ill-defined, with distinct conceptual labels, such as external vs. mutual, symmetrical vs. asymmetrical, metrical vs. non-metrical, within-persons and between-person, and physical vs. cognitive entrainment. The boundaries between entrainment, resonance, and synchronization are also not always very clear. There is, as such, a need for a broadened approach to entrainment, taking as a starting point the concept of oscillators that interact with each other in a continuous and ongoing way, and relying on the theoretical framework of interaction dynamics and the concept of adaptation. Entrainment, in this broadened view, is seen as an adaptive process that accommodates to the music under the influence of both the attentional direction of the listener and the configurations of the sounding stimuli.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"135 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124206891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards Universal Industrial Augmented Reality: Implementing a Modular IAR System to Support Assembly Processes","authors":"Detlef Gerhard, Matthias Neges, Jan Luca Siewert, M. Wolf","doi":"10.3390/mti7070065","DOIUrl":"https://doi.org/10.3390/mti7070065","url":null,"abstract":"While Industrial Augmented Reality (IAR) has many applications across the whole product lifecycle, most IAR applications today are custom-built for specific use-cases in practice. This contribution builds upon a scoping literature review of IAR data representations to present a modern, modular IAR architecture. The individual modules of the presented approach are either responsible for user interface and user interaction or for data processing. They are use-case neutral and independent of each other, while communicating through a strictly separated application layer. To demonstrate the architecture, this contribution presents an assembly process that is supported once with a pick-to-light system and once using in situ projections. Both are implemented on top of the novel architecture, allowing most of the work on the individual models to be reused. This IAR architecture, based on clearly separated modules with defined interfaces, particularly allows small companies with limited personnel resources to adapt IAR for their specific use-cases more easily than developing single-use applications from scratch.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122605680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"From Earlier Exploration to Advanced Applications: Bibliometric and Systematic Review of Augmented Reality in the Tourism Industry (2002-2022)","authors":"Mohamed Zaifri, H. Khalloufi, Fatima-Zahra Kaghat, Ahmed Azough, Khalid Alaoui Zidani","doi":"10.3390/mti7070064","DOIUrl":"https://doi.org/10.3390/mti7070064","url":null,"abstract":"Augmented reality has emerged as a transformative technology, with the potential to revolutionize the tourism industry. Nonetheless, there is a scarcity of studies tracing the progression of AR and its application in tourism, from early exploration to recent advancements. This study aims to provide a comprehensive overview of the evolution, contexts, and design elements of AR in tourism over the period (2002–2022), offering insights for further progress in this domain. Employing a dual-method approach, a bibliometric analysis was conducted on 861 articles collected from the Scopus and Web of Science databases, to investigate the evolution of AR research over time and across countries, and to identify the main contexts of the utilization of AR in tourism. In the second part of our study, a systematic content analysis was conducted, focusing on a subset of 57 selected studies that specifically employed AR systems in various tourism situations. Through this analysis, the most commonly utilized AR design components, such as tracking systems, AR devices, tourism settings, and virtual content were summarized. Furthermore, we explored how these components were integrated to enhance the overall tourism experience. The findings reveal a growing trend in research production, led by Europe and Asia. Key contexts of AR applications in tourism encompass cultural heritage, mobile AR, and smart tourism, with emerging topics such as artificial intelligence (AI), big data, and COVID-19. Frequently used AR design components comprise mobile devices, marker-less tracking systems, outdoor environments, and visual overlays. Future research could involve optimizing AR experiences for users with disabilities, supporting multicultural experiences, integrating AI with big data, fostering sustainability, and remote virtual tourism. This study contributes to the ongoing discourse on the role of AR in shaping the future of tourism in the post COVID-19 era, by providing valuable insights for researchers, practitioners, and policymakers in the tourism industry.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126460737","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Mid-Air Gestural Interaction with a Large Fogscreen","authors":"V. Remizova, Antti Sand, I. Mackenzie, O. Špakov, Katariina Nyyssönen, I. Rakkolainen, A. Kylliäinen, Veikko Surakka, Y. Gizatdinova","doi":"10.3390/mti7070063","DOIUrl":"https://doi.org/10.3390/mti7070063","url":null,"abstract":"Projected walk-through fogscreens have been created, but there is little research on the evaluation of the interaction performance with fogscreens. The present study investigated mid-air hand gestures for interaction with a large fogscreen. Participants (N = 20) selected objects from a fogscreen using tapping and dwell-based gestural techniques, with and without vibrotactile/haptic feedback. In terms of Fitts’ law, the throughput was about 1.4 bps to 2.6 bps, suggesting that gestural interaction with a large fogscreen is a suitable and effective input method. Our results also suggest that tapping without haptic feedback has good performance and potential for interaction with a fogscreen, and that tactile feedback is not necessary for effective mid-air interaction. These findings have implications for the design of gestural interfaces suitable for interaction with fogscreens.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116104011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Are Drivers Allowed to Sleep? Sleep Inertia Effects Drivers' Performance after Different Sleep Durations in Automated Driving","authors":"Doreen Schwarze, F. Diederichs, Lukas Weiser, H. Widlroither, Rolf Verhoeven, M. Rötting","doi":"10.3390/mti7060062","DOIUrl":"https://doi.org/10.3390/mti7060062","url":null,"abstract":"Higher levels of automated driving may offer the possibility to sleep in the driver’s seat in the car, and it is foreseeable that drivers will voluntarily or involuntarily fall asleep when they do not need to drive. Post-sleep performance impairments due to sleep inertia, a brief period of impaired cognitive performance after waking up, is a potential safety issue when drivers need to take over and drive manually. The present study assessed whether sleep inertia has an effect on driving and cognitive performance after different sleep durations. A driving simulator study with n = 13 participants was conducted. Driving and cognitive performance were analyzed after waking up from a 10–20 min sleep, a 30–60 min sleep, and after resting without sleep. The study’s results indicate that a short sleep duration does not reliably prevent sleep inertia. After the 10–20 min sleep, cognitive performance upon waking up was decreased, but the sleep inertia impairment faded within 15 min. Although the driving parameters showed no significant difference between the conditions, participants subjectively felt more tired after both sleep durations compared to resting. The small sample size of 13 participants, tested in a within-design, may have prevented medium and small effects from becoming significant. In our study, take-over was offered without time pressure, and take-over times ranged from 3.15 min to 4.09 min after the alarm bell, with a mean value of 3.56 min in both sleeping conditions. The results suggest that daytime naps without previous sleep deprivation result in mild and short-term impairments. Further research is recommended to understand the severity of impairments caused by different intensities of sleep inertia.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"156 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116957418","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Social Cohesion in Interactive Digital Heritage Experiences","authors":"S. Pescarin, Vanessa Bonanno, A. Marasco","doi":"10.3390/mti7060061","DOIUrl":"https://doi.org/10.3390/mti7060061","url":null,"abstract":"Presently, social cohesion (SC) is a priority at different levels. Cultural heritage is an ideal context to promote SC through interactive digital technologies designed to engage groups of visitors. The purpose of the present study is to identify how to design digital heritage applications for SC and how to measure it. The results are based on the design of a cultural probe kit used to identify the design elements on top of which a collaborative and hybrid prototype, the Brancacci POV, was developed. Here, we analysed the results of this prototype, which included 107 visitors with respective groups of 5 participants and guided by an expert. From this analysis, the possibility of strengthening SC when collaborative tasks are included emerged. Additionally, it appeared to be possible to shorten the distance between citizens and cultural institutions if “mediated dialogue” approaches were adopted and if focus, motivation, trust and “in-group” perception of inclusion emerge when digital heritage experiences were set in intimate and quiet environments.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"271 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125829163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Location-Based Game for Thought-Provoking Evacuation Training","authors":"Hiroyuki Mitsuhara, Chie Tanimura, Junko Nemoto, M. Shishibori","doi":"10.3390/mti7060059","DOIUrl":"https://doi.org/10.3390/mti7060059","url":null,"abstract":"Participation in evacuation training can aid survival in the event of an unpredictable disaster, such as an earthquake. However, conventional evacuation training is not well designed for provoking critical thinking in participants regarding the processes involved in a successful evacuation. To realize thought-provoking evacuation training, we developed a location-based game that presents digital materials that express disaster situations corresponding to locations or times preset in a scenario and providing scenario-based multi-ending as the game element. The developed game motivates participants to take decisions by providing high situational and audiovisual realism. In addition, the game encourages the participants to think objectively about the evacuation process by working together with a reflection-support system. We practiced thought-provoking evacuation training with fifth-grade students, focusing on tsunami evacuation and lifesaving-related moral dilemmas. In this practice, we observed that the participants took decisions as if they were dealing with actual disaster situations and objectively thought about the evacuation process by reflecting on their decisions. Meanwhile, we found that lifesaving-related moral dilemmas are difficult to address in evacuation training.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125295104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the Effectiveness of Using Virtual Reality to View BIM Metadata in Architectural Design Reviews for Healthcare","authors":"Emma Buchanan, G. Loporcaro, S. Lukosch","doi":"10.3390/mti7060060","DOIUrl":"https://doi.org/10.3390/mti7060060","url":null,"abstract":"This article reports on a study that assessed whether Virtual Reality (VR) can be used to display Building Information Modelling (BIM) metadata alongside spatial data in a virtual environment, and by doing so determine if it increases the effectiveness of the design review by improving participants’ understanding of the design. Previous research has illustrated the potential for VR to enhance design reviews, especially the ability to convey spatial information, but there has been limited research into how VR can convey additional BIM metadata. A user study with 17 healthcare professionals assessed participants’ performances and preferences for completing design reviews in VR or using a traditional design review system of PDF drawings and a 3D model. The VR condition had a higher task completion rate, a higher SUS score and generally faster completion times. VR increases the effectiveness of a design review conducted by healthcare professionals.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133400562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development and Evaluation of a Mobile Application with Augmented Reality for Guiding Visitors on Hiking Trails","authors":"Rute Silva, R. Jesus, P. Jorge","doi":"10.3390/mti7060058","DOIUrl":"https://doi.org/10.3390/mti7060058","url":null,"abstract":"Tourism on the island of Santa Maria, Azores, has been increasing due to its characteristics in terms of biodiversity and geodiversity. This island has several hiking trails; the available information can be consulted in pamphlets and physical placards, whose maintenance and updating is difficult and expensive. Thus, the need to improve the visitors’ experience arises, in this case, by using the technological means currently available to everyone: a smartphone. This paper describes the development and evaluation of the user experience of a mobile application for guiding visitors on said hiking trails, as well as the design principles and main issues observed during this process. The application is based on an augmented reality interaction model providing visitors with an interactive and recreational experience through Augmented Reality in outdoor environments (without additional marks in the physical space and using georeferenced information), helping in navigation during the route and providing updated information with easy maintenance. For the design and evaluation of the application, two studies were carried out with users on-site (Santa Maria, Azores). The first had 77 participants, to analyze users and define the application’s characteristics, and the second had 10 participants to evaluate the user experience. The feedback from participants was obtained through questionnaires. In these questionnaires, an average SUS (System Usability Scale) score of 83 (excellent) and positive results in the UEQ (User Experience Questionnaire) were obtained.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130513909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}