Journal on Multimodal User Interfaces最新文献_第8页

Predicting multimodal presentation skills based on instance weighting domain adaptation 基于实例加权域自适应的多模态表达技巧预测

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-02-18 DOI: 10.1007/s12193-021-00367-x

Yutaro Yagi, Shogo Okada, Shota Shiobara, Sota Sugimura

{"title":"Predicting multimodal presentation skills based on instance weighting domain adaptation","authors":"Yutaro Yagi, Shogo Okada, Shota Shiobara, Sota Sugimura","doi":"10.1007/s12193-021-00367-x","DOIUrl":"https://doi.org/10.1007/s12193-021-00367-x","url":null,"abstract":"Presentation skills assessment is one of the central challenges of multimodal modeling. Presentation skills are composed of verbal and nonverbal skill components, but because people demonstrate their presentation skills in a variety of manners, the observed multimodal features vary widely. Due to the differences in features, when test data samples are generated on different training data sample distributions, in many cases, the prediction accuracy of the skills degrades. In machine learning theory, this problem in which training (source) data are biased is known as instance selection bias or covariate shift. To solve this problem, this paper presents an instance weighting adaptation method that is applied to estimate the presentation skills of each participant from multimodal (verbal and nonverbal) features. For this purpose, we collect a novel multimodal presentation dataset that includes audio signal data, body motion sensor data, and text data of the speech content for participants observed in 58 presentation sessions. The dataset also includes both verbal and nonverbal presentation skills, which are assessed by two external experts from a human resources department. We extract multimodal features, such as spoken utterances, acoustic features, and the amount of body motion, to estimate the presentation skills. We propose two approaches, early fusing and late fusing, for the regression models based on multimodal instance weighting adaptation. The experimental results show that the early fusing regression model with instance weighting adaptation achieved (rho =0.39) for the Pearson correlation, which presents the regression accuracy for the clarity of presentation goal elements. In the maximum case, the accuracy (correlation coefficient) is improved from (-0.34) to +0.35 by instance weighting adaptation.","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"33 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2021-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Facial expression and action unit recognition augmented by their dependencies on graph convolutional networks 基于图卷积网络增强的面部表情和动作单元识别

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-26 DOI: 10.1007/s12193-020-00363-7

Jun He, Xiaocui Yu, Bo Sun, Lejun Yu

引用次数: 7

Neighborhood based decision theoretic rough set under dynamic granulation for BCI motor imagery classification 动态粒化下基于邻域决策理论粗糙集的脑机接口运动图像分类

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-25 DOI: 10.1007/s12193-020-00358-4

K. Renuga Devi, H. Hannah Inbarani

引用次数: 11

Comparing mind perception in strategic exchanges: human-agent negotiation, dictator and ultimatum games 战略交流中的心理感知比较:人-代理谈判、独裁者与最后通牒博弈

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-21 DOI: 10.1007/s12193-020-00356-6

Minha Lee, Gale M. Lucas, J. Gratch

引用次数: 13

A novel focus encoding scheme for addressee detection in multiparty interaction using machine learning algorithms 一种基于机器学习算法的多方交互中地址检测的焦点编码方案

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-17 DOI: 10.1007/s12193-020-00361-9

Usman Malik, Mukesh Barange, Julien Saunier, A. Pauchet

引用次数: 3

PLAAN: Pain Level Assessment with Anomaly-detection based Network PLAAN:基于异常检测网络的疼痛水平评估

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-06 DOI: 10.1007/s12193-020-00362-8

Yi Li, Shreya Ghosh, Jyoti Joshi

引用次数: 6

Internet-based tailored virtual human health intervention to promote colorectal cancer screening: design guidelines from two user studies 基于互联网的定制虚拟人类健康干预以促进结直肠癌筛查:来自两项用户研究的设计指南

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-02 DOI: 10.1007/s12193-020-00357-5

Mohan S Zalake, F. Tavassoli, Kyle A. Duke, T. George, François Modave, J. Neil, Janice L. Krieger, B. Lok

引用次数: 4

Words of encouragement: how praise delivered by a social robot changes children’s mindset for learning 鼓励的话语:社交机器人的赞美如何改变孩子的学习心态

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2020-11-24 DOI: 10.1007/s12193-020-00353-9

Daniel P. Davison, F. Wijnen, V. Charisi, J. van der Meij, D. Reidsma, V. Evers

引用次数: 14

An audiovisual interface-based drumming system for multimodal human–robot interaction 基于视听界面的多模态人机交互击鼓系统

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2020-11-13 DOI: 10.1007/s12193-020-00352-w

G. Ince, R. Yorganci, A. Ozkul, Taha Berkay Duman, Hatice Köse

引用次数: 7

Virtual agents as supporting media for scientific presentations 虚拟代理作为科学演示的支持媒体

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2020-11-06 DOI: 10.1007/s12193-020-00350-y

T. Bickmore, Everlyne Kimani, Ameneh Shamekhi, Prasanth Murali, Dhaval Parmar, H. Trinh

引用次数: 9