Proceedings of the 2020 International Conference on Multimodal Interaction最新文献_第8页

Finally on Par?! Multimodal and Unimodal Interaction for Open Creative Design Tasks in Virtual Reality 终于平起平坐了?!虚拟现实中开放式创意设计任务的多模态和单模态交互

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3418850

C. Zimmerer, Erik Wolf, Sara Wolf, Martin Fischbach, Jean-Luc Lugrin, Marc Erich Latoschik

{"title":"Finally on Par?! Multimodal and Unimodal Interaction for Open Creative Design Tasks in Virtual Reality","authors":"C. Zimmerer, Erik Wolf, Sara Wolf, Martin Fischbach, Jean-Luc Lugrin, Marc Erich Latoschik","doi":"10.1145/3382507.3418850","DOIUrl":"https://doi.org/10.1145/3382507.3418850","url":null,"abstract":"Multimodal Interfaces (MMIs) have been considered to provide promising interaction paradigms for Virtual Reality (VR) for some time. However, they are still far less common than unimodal interfaces (UMIs). This paper presents a summative user study comparing an MMI to a typical UMI for a design task in VR. We developed an application targeting creative 3D object manipulations, i.e., creating 3D objects and modifying typical object properties such as color or size. The associated open user task is based on the Torrence Tests of Creative Thinking. We compared a synergistic multimodal interface using speech-accompanied pointing/grabbing gestures with a more typical unimodal interface using a hierarchical radial menu to trigger actions on selected objects. Independent judges rated the creativity of the resulting products using the Consensual Assessment Technique. Additionally, we measured the creativity-promoting factors flow, usability, and presence. Our results show that the MMI performs on par with the UMI in all measurements despite its limited flexibility and reliability. These promising results demonstrate the technological maturity of MMIs and their potential to extend traditional interaction techniques in VR efficiently.","PeriodicalId":402394,"journal":{"name":"Proceedings of the 2020 International Conference on Multimodal Interaction","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121071096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Influence of Electric Taste, Smell, Color, and Thermal Sensory Modalities on the Liking and Mediated Emotions of Virtual Flavor Perception 电味觉、嗅觉、颜色和热感觉方式对虚拟味觉的喜爱和介导情绪的影响

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3418862

Nimesha Ranasinghe, Meetha Nesam James, Michael Gecawicz, Jonathan Bland, David Smith

{"title":"Influence of Electric Taste, Smell, Color, and Thermal Sensory Modalities on the Liking and Mediated Emotions of Virtual Flavor Perception","authors":"Nimesha Ranasinghe, Meetha Nesam James, Michael Gecawicz, Jonathan Bland, David Smith","doi":"10.1145/3382507.3418862","DOIUrl":"https://doi.org/10.1145/3382507.3418862","url":null,"abstract":"Little is known about the influence of various sensory modalities such as taste, smell, color, and thermal, towards perceiving simulated flavor sensations, let alone their influence on people's emotions and liking. Although flavor sensations are essential in our daily experiences and closely associated with our memories and emotions, the concept of flavor and the emotions caused by different sensory modalities are not thoroughly integrated into Virtual and Augmented Reality technologies. Hence, this paper presents 1) an interactive technology to simulate different flavor sensations by overlaying taste (via electrical stimulation on the tongue), smell (via micro air pumps), color (via RGB Lights), and thermal (via Peltier elements) sensations on plain water, and 2) a set of experiments to investigate a) the influence of different sensory modalities on the perception and liking of virtual flavors and b) varying emotions mediated through virtual flavor sensations. Our findings reveal that the participants perceived and liked various stimuli configurations and mostly associated them with positive emotions while highlighting important avenues for future research.","PeriodicalId":402394,"journal":{"name":"Proceedings of the 2020 International Conference on Multimodal Interaction","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128567423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Fusical: Multimodal Fusion for Video Sentiment Fusical:视频情感的多模态融合

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3417966

Bo Jin, L. Abdelrahman, C. Chen, Amil Khanzada

引用次数: 3

Supporting Instructors to Provide Emotional and Instructional Scaffolding for English Language Learners through Biosensor-based Feedback 支持教师通过基于生物传感器的反馈为英语学习者提供情感和教学脚手架

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3421159

Heera Lee

{"title":"Supporting Instructors to Provide Emotional and Instructional Scaffolding for English Language Learners through Biosensor-based Feedback","authors":"Heera Lee","doi":"10.1145/3382507.3421159","DOIUrl":"https://doi.org/10.1145/3382507.3421159","url":null,"abstract":"Delivering a presentation has been reported as one of the most anxiety-provoking tasks faced by English Language Learners. Researchers suggest that instructors should be more aware of the learners' emotional states to provide appropriate emotional and instructional scaffolding to improve their performance when presenting. Despite the critical role of instructors in perceiving the emotional states among English language learners, it can be challenging to do this solely by observing the learners? facial expressions, behaviors, and their limited verbal expressions due to language and cultural barriers. To address the ambiguity and inconsistency in interpreting the emotional states of the students, this research focuses on identifying the potential of using biosensor-based feedback of learners to support instructors. A novel approach has been adopted to classify the intensity and characteristics of public speaking anxiety and foreign language anxiety among English language learners and to provide tailored feedback to instructors while supporting teaching and learning. As part of this work, two further studies were proposed. The first study was designed to identify educators' needs for solutions providing emotional and instructional support. The second study aims to evaluate a resulting prototype as a view of instructors to offer tailored emotional and instructional scaffolding to students. The contribution of these studies includes the development of guidance in using biosensor-based feedback that will assist English language instructors in teaching and identifying the students' anxiety levels and types while delivering a presentation.","PeriodicalId":402394,"journal":{"name":"Proceedings of the 2020 International Conference on Multimodal Interaction","volume":"83 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126220980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Touch Recognition with Attentive End-to-End Model 基于细心端到端模型的触摸识别

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3418834

Wail El Bani, M. Chetouani

引用次数: 0

LDNN: Linguistic Knowledge Injectable Deep Neural Network for Group Cohesiveness Understanding 面向群体内聚性理解的语言知识注入深度神经网络

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3418830

Yanan Wang, Jianming Wu, Jinfa Huang, Gen Hattori, Y. Takishima, Shinya Wada, Rui Kimura, Jie Chen, Satoshi Kurihara

{"title":"LDNN: Linguistic Knowledge Injectable Deep Neural Network for Group Cohesiveness Understanding","authors":"Yanan Wang, Jianming Wu, Jinfa Huang, Gen Hattori, Y. Takishima, Shinya Wada, Rui Kimura, Jie Chen, Satoshi Kurihara","doi":"10.1145/3382507.3418830","DOIUrl":"https://doi.org/10.1145/3382507.3418830","url":null,"abstract":"Group cohesiveness reflects the level of intimacy that people feel with each other, and the development of a dialogue robot that can understand group cohesiveness will lead to the promotion of human communication. However, group cohesiveness is a complex concept that is difficult to predict based only on image pixels. Inspired by the fact that humans intuitively associate linguistic knowledge accumulated in the brain with the visual images they see, we propose a linguistic knowledge injectable deep neural network (LDNN) that builds a visual model (visual LDNN) for predicting group cohesiveness that can automatically associate the linguistic knowledge hidden behind images. LDNN consists of a visual encoder and a language encoder, and applies domain adaptation and linguistic knowledge transition mechanisms to transform linguistic knowledge from a language model to the visual LDNN. We train LDNN by adding descriptions to the training and validation sets of the Group AFfect Dataset 3.0 (GAF 3.0), and test the visual LDNN without any description. Comparing visual LDNN with various fine-tuned DNN models and three state-of-the-art models in the test set, the results demonstrate that the visual LDNN not only improves the performance of the fine-tuned DNN model leading to an MSE very similar to the state-of-the-art model, but is also a practical and efficient method that requires relatively little preprocessing. Furthermore, ablation studies confirm that LDNN is an effective method to inject linguistic knowledge into visual models.","PeriodicalId":402394,"journal":{"name":"Proceedings of the 2020 International Conference on Multimodal Interaction","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127963772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Multi-modal Fusion Using Spatio-temporal and Static Features for Group Emotion Recognition 基于时空和静态特征的多模态融合群体情绪识别

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3417971

Mo Sun, Jian Li, Hui Feng, Wei Gou, Haifeng Shen, Jian-Bo Tang, Yi Yang, Jieping Ye

引用次数: 9

The AI-Medic: A Multimodal Artificial Intelligent Mentor for Trauma Surgery AI-Medic:创伤外科的多模式人工智能导师

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3421167

Edgar Rojas-Muñoz, K. Couperus, J. Wachs

引用次数: 0

Hand-eye Coordination for Textual Difficulty Detection in Text Summarization 文本摘要中文本难度检测的手眼协调

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3418831

Jun Wang, G. Ngai, H. Leong

引用次数: 0

Multimodal Interaction in Psychopathology 精神病理学中的多模态相互作用

Proceedings of the 2020 International Conference on Multimodal Interaction Pub Date : 2020-10-21 DOI: 10.1145/3382507.3419751

Itir Onal Ertugrul, J. Cohn, Hamdi Dibeklioğlu

引用次数: 0