Multimodal Technol. Interact.最新文献

{"title":"Does Augmented Reality Help to Understand Chemical Phenomena during Hands-On Experiments?-Implications for Cognitive Load and Learning","authors":"Hendrik Peeters, Sebastian Habig, Sabine Fechner","doi":"10.3390/mti7020009","DOIUrl":"https://doi.org/10.3390/mti7020009","url":null,"abstract":"Chemical phenomena are only observable on a macroscopic level, whereas they are explained by entities on a non-visible level. Students often demonstrate limited ability to link these different levels. Augmented reality (AR) offers the possibility to increase contiguity by embedding virtual models into hands-on experiments. Therefore, this paper presents a pre- and post-test study investigating how learning and cognitive load are influenced by AR during hands-on experiments. Three comparison groups (AR, animation and filmstrip), with a total of N = 104 German secondary school students, conducted and explained two hands-on experiments. Whereas the AR group was allowed to use an AR app showing virtual models of the processes on the submicroscopic level during the experiments, the two other groups were provided with the same dynamic or static models after experimenting. Results indicate no significant learning gain for the AR group in contrast to the two other groups. The perceived intrinsic cognitive load was higher for the AR group in both experiments as well as the extraneous load in the second experiment. It can be concluded that AR could not unleash its theoretically derived potential in the present study.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133087750","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Acknowledgment to the Reviewers of MTI in 2022","authors":"Mti Editorial Office","doi":"10.3390/mti7020007","DOIUrl":"https://doi.org/10.3390/mti7020007","url":null,"abstract":"High-quality academic publishing is built on rigorous peer review Multimodal Technologies and Interaction (MTI) was able to uphold its high standards for published papers due to the outstanding efforts of our reviewers [...]","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"83 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134547897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multimodal Communication and Peer Interaction during Equation-Solving Sessions with and without Tangible Technologies","authors":"Daranee Lehtonen, Jorma Joutsenlahti, P. Perkkilä","doi":"10.3390/mti7010006","DOIUrl":"https://doi.org/10.3390/mti7010006","url":null,"abstract":"Despite the increasing use of technologies in the classroom, there are concerns that technology-enhanced learning environments may hinder students’ communication and interaction. In this study, we investigated how tangible technologies can enhance students’ multimodal communication and interaction during equation-solving pair work compared to working without such technologies. A tangible app for learning equation solving was developed and tested in fourth- and fifth-grade classrooms with two class teachers and 24 students. Video data of the interventions were analysed using deductive and inductive content analysis. Coded data were also quantified for quantitative analysis. Additionally, teacher interview data were used to compare and contrast the findings. The findings showed that the tangible app better promoted students’ multimodal communication and peer interaction than working only with paper and pencil. When working in pairs, tangible-app students interacted with one another much more often and in more ways than their paper-and-pencil peers. The implications of this study are discussed in terms of its contributions to research on tangible technologies for learning, educational technology development, and the use of tangibles in classrooms to support students’ multimodal communication and peer interaction.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117314847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Decision-Support System to Analyse Customer Satisfaction Applied to a Tourism Transport Service","authors":"C. Ramos, P. Cardoso, Hortênsio C. L. Fernandes, J. Rodrigues","doi":"10.3390/mti7010005","DOIUrl":"https://doi.org/10.3390/mti7010005","url":null,"abstract":"Due to the perishable nature of tourist products, which impacts supply and demand, the possibility of analysing the relationship between customers’ satisfaction and service quality can contribute to increased revenues. Machine learning techniques allow the analysis of how these services can be improved or developed and how to reach new markets, and look for the emergence of ideas to innovate and improve interaction with the customer. This paper presents a decision-support system for analysing consumer satisfaction, based on consumer feedback from the customer’s experience when transported by a transfer company, in the present case working in the Algarve region, Portugal. The results show how tourists perceive the service and which factors influence their level of satisfaction and sentiment. One of the results revealed that the first impression associated with good news is what creates the most value in the experience, i.e., “first impressions matter”.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134520558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Value of Context-Based Interface Prototyping for the Autonomous Vehicle Domain: A Method Overview","authors":"Lukas A. Flohr, D. Wallach","doi":"10.3390/mti7010004","DOIUrl":"https://doi.org/10.3390/mti7010004","url":null,"abstract":"Before autonomous vehicles (AVs; SAE levels 4 and 5) become broadly available, acceptance challenges such as trust and safety concerns must be overcome. In the development of appropriate HMIs that will tackle these challenges, physical and social context play essential roles. Contextual factors thus need to be considered in early prototyping stages. Based on a qualitative semi-systematic literature review and knowledge from our research, this paper elaborates on the value of context-based interface prototyping in the AV domain. It provides a comprehensive overview and a discussion of applicable methods, including physical lab-based prototyping (mock-up, ride simulation with virtual and mixed reality, and immersive video), social context simulation (actors, enactment, items and props, and sound), wizard-of-oz, and experimental vehicles. Finally, the paper discusses factors affecting the impact of prototyping and derives recommendations for the application of prototyping methods in future AV studies.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123937688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multimodal Augmented Reality Applications for Training of Traffic Procedures in Aviation","authors":"Birgit Moesl, Harald Schaffernak, Wolfgang Vorraber, R. Braunstingl, I. Koglbauer","doi":"10.3390/mti7010003","DOIUrl":"https://doi.org/10.3390/mti7010003","url":null,"abstract":"Mid-air collision is one of the top safety risks in general aviation. This study addresses the description and experimental assessment of multimodal Augmented Reality (AR) applications for training of traffic procedures in accordance with Visual Flight Rules (VFR). AR has the potential to complement the conventional flight instruction by bridging the gap between theory and practice, and by releasing students’ time and performance pressure associated with a limited simulator time. However, it is critical to assess the impact of AR in the specific domain and to identify any potential negative learning transfer. Multimodal AR applications were developed to address various areas of training: guidance and feedback for the correct scanning pattern, estimation if an encountering traffic is on collision course and application of the relevant rules. The AR applications also provided performance feedback for collision detection, avoidance and priority decisions. The experimental assessment was conducted with 59 trainees (28 women, 31 men) assigned to an experimental group (AR training) and a control group (simulator training). The results of tests without AR in the flight simulator show that the group that trained with AR obtained similar levels of performance like the control group. There was no negative training effect of AR on trainees’ performance, workload, situational awareness, emotion nor motivation. After training the tasks were perceived as less challenging, the accuracy of collision detection has improved, and the trainees reported less intense negative emotions and fear of failure. Furthermore, a scanning pattern test in AR showed that the AR training group performed the scanning pattern significantly better than the control group. In addition, there was a significant gender effect on emotion, motivation and preferences for AR features, but not on performance. Women liked the voice interaction with AR and the compass hologram more than men. Men liked the traffic holograms and the AR projection field more than women. These results are important because they provide experimental evidence for the benefits of multimodal AR applications that could be used complementary to the flight simulator training.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"110 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125632057","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"\"AR The Gods of Olympus\": Design and Pilot Evaluation of an Augmented Reality Educational Game for Greek Mythology","authors":"Evangelos Ventoulis, S. Xinogalos","doi":"10.3390/mti7010002","DOIUrl":"https://doi.org/10.3390/mti7010002","url":null,"abstract":"Teaching and learning theoretical subjects, such as History, although important, is considered by many students to be non-appealing. Alternative teaching approaches include the use of educational games and augmented reality (AR) applications, or more recently, AR educational games. Such games are considered to increase students’ interest for the subject and lead to better learning outcomes. However, studies about the use of AR educational games in the classroom are sparse and further research is necessary. In this article, we present an AR-enhanced educational game for teaching History (Greek Mythology) to 3rd grade Primary school students in Greece. The game, called “AR The Gods of Olympus” consists of three mini games: an AR game with the gods/goddesses of Olympus using narration; a memory game with cards depicting the gods and their symbols; and a quiz game. In order to study the effectiveness of the game and students’ experience and perceptions on it, a study was carried out with primary school students that used the game in classroom. The study utilized a pre-/post-test design, a brief questionnaire based on the MEEGA+ model for evaluating educational games, and observation of students during game playing. Students’ performance was improved after playing the game but the difference was not statistically significant, while the game was positively perceived by students. Especially the AR mini game raised students’ interest and as the students themselves stated helped them “learn while playing”.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122803837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Co-Design within and between Communities in Cultural Heritage: Current and Open Questions","authors":"Laura A. Maye, C. Claisse","doi":"10.3390/mti7010001","DOIUrl":"https://doi.org/10.3390/mti7010001","url":null,"abstract":"In this Special Issue, we explore emerging theories, methods, and case studies focusing on co-design activities within and between communities in cultural heritage contexts. [...]","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115298839","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing Dynamic Audio Navigation UIs to Pinpoint Elements in Tactile Graphics","authors":"Gaspar Ramôa, Vincent Schmidt, P. König","doi":"10.3390/mti6120113","DOIUrl":"https://doi.org/10.3390/mti6120113","url":null,"abstract":"Access to complex graphical information is essential when connecting blind and visually impaired (BVI) people with the world. Tactile graphics readers enable access to graphical data through audio-tactile user interfaces (UIs), but these have yet to mature. A challenging task for blind people is locating specific elements–areas in detailed tactile graphics. To this end, we developed three audio navigation UIs that dynamically guide the user’s hand to a specific position using audio feedback. One is based on submarine sonar sounds, another relies on the target’s coordinate plan x and y-axis, and the last uses direct voice instructions. The UIs were implemented in the Tactonom Reader device, a new tactile graphic reader that enhances swell paper graphics with pinpointed audio explanations. To evaluate the effectiveness of the three different dynamic navigation UIs, we conducted a within-subject usability test that involved 13 BVI participants. Beyond comparing the effectiveness of the different UIs, we observed and recorded the interaction of the visually impaired participants with the different navigation UI to further investigate their behavioral patterns during the interaction. We observed that user interfaces that required the user to move their hand in a straight direction were more likely to provoke frustration and were often perceived as challenging for blind and visually impaired people. The analysis revealed that the voice-based navigation UI guides the participant the fastest to the target and does not require prior training. This suggests that a voice-based navigation strategy is a promising approach for designing an accessible user interface for the blind.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"28 9","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131539582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Machine Learning Approach to Identify the Preferred Representational System of a Person","authors":"Mohammad Hossein Amirhosseini, J. Wall","doi":"10.3390/mti6120112","DOIUrl":"https://doi.org/10.3390/mti6120112","url":null,"abstract":"Whenever people think about something or engage in activities, internal mental processes will be engaged. These processes consist of sensory representations, such as visual, auditory, and kinesthetic, which are constantly being used, and they can have an impact on a person’s performance. Each person has a preferred representational system they use most when speaking, learning, or communicating, and identifying it can explain a large part of their exhibited behaviours and characteristics. This paper proposes a machine learning-based automated approach to identify the preferred representational system of a person that is used unconsciously. A novel methodology has been used to create a specific labelled conversational dataset, four different machine learning models (support vector machine, logistic regression, random forest, and k-nearest neighbour) have been implemented, and the performance of these models has been evaluated and compared. The results show that the support vector machine model has the best performance for identifying a person’s preferred representational system, as it has a better mean accuracy score compared to the other approaches after the performance of 10-fold cross-validation. The automated model proposed here can assist Neuro Linguistic Programming practitioners and psychologists to have a better understanding of their clients’ behavioural patterns and the relevant cognitive processes. It can also be used by people and organisations in order to achieve their goals in personal development and management. The two main knowledge contributions in this paper are the creation of the first labelled dataset for representational systems, which is now publicly available, and the use of machine learning techniques for the first time to identify a person’s preferred representational system in an automated way.","PeriodicalId":408374,"journal":{"name":"Multimodal Technol. Interact.","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127811465","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}