Journal on Multimodal User Interfaces最新文献_第8页

An expert-model and machine learning hybrid approach to predicting human-agent negotiation outcomes in varied data 一种专家模型和机器学习混合方法在不同数据中预测人类代理谈判结果

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-03-03 DOI: 10.1007/s12193-021-00368-w

Johnathan Mell, Markus Beissinger, Jonathan Gratch

{"title":"An expert-model and machine learning hybrid approach to predicting human-agent negotiation outcomes in varied data","authors":"Johnathan Mell, Markus Beissinger, Jonathan Gratch","doi":"10.1007/s12193-021-00368-w","DOIUrl":"https://doi.org/10.1007/s12193-021-00368-w","url":null,"abstract":"We present the results of a machine-learning approach to the analysis of several human-agent negotiation studies. By combining expert knowledge of negotiating behavior compiled over a series of empirical studies with neural networks, we show that a hybrid approach to parameter selection yields promise for designing more effective and socially intelligent agents. Specifically, we show that a deep feedforward neural network using a theory-driven three-parameter model can be effective in predicting negotiation outcomes. Furthermore, it outperforms other expert-designed models that use more parameters, as well as those using other techniques (such as linear regression models or boosted decision trees). In a follow-up study, we show that the most successful models change as the dataset size increases and the prediction targets change, and show that boosted decision trees may not be suitable for the negotiation domain. We anticipate these results will have impact for those seeking to combine extensive domain knowledge with more automated approaches in human-computer negotiation. Further, we show that this approach can be a stepping stone from purely exploratory research to targeted human-behavioral experimentation. Through our approach, areas of social artificial intelligence that have historically benefited from expert knowledge and traditional AI approaches can be combined with more recent proven-effective machine learning algorithms.","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"34 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2021-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

MUMBAI: multi-person, multimodal board game affect and interaction analysis dataset 多人、多模式棋盘游戏影响和互动分析数据集

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-02-22 DOI: 10.1007/s12193-021-00364-0

Metehan Doyran, Arjan Schimmel, P. Baki, Kübra Ergin, Batikan Türkmen, A. A. Salah, S. Bakkes, Heysem Kaya, R. Poppe, A. A. Salah

引用次数: 19

Predicting multimodal presentation skills based on instance weighting domain adaptation 基于实例加权域自适应的多模态表达技巧预测

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-02-18 DOI: 10.1007/s12193-021-00367-x

Yutaro Yagi, Shogo Okada, Shota Shiobara, Sota Sugimura

{"title":"Predicting multimodal presentation skills based on instance weighting domain adaptation","authors":"Yutaro Yagi, Shogo Okada, Shota Shiobara, Sota Sugimura","doi":"10.1007/s12193-021-00367-x","DOIUrl":"https://doi.org/10.1007/s12193-021-00367-x","url":null,"abstract":"Presentation skills assessment is one of the central challenges of multimodal modeling. Presentation skills are composed of verbal and nonverbal skill components, but because people demonstrate their presentation skills in a variety of manners, the observed multimodal features vary widely. Due to the differences in features, when test data samples are generated on different training data sample distributions, in many cases, the prediction accuracy of the skills degrades. In machine learning theory, this problem in which training (source) data are biased is known as instance selection bias or covariate shift. To solve this problem, this paper presents an instance weighting adaptation method that is applied to estimate the presentation skills of each participant from multimodal (verbal and nonverbal) features. For this purpose, we collect a novel multimodal presentation dataset that includes audio signal data, body motion sensor data, and text data of the speech content for participants observed in 58 presentation sessions. The dataset also includes both verbal and nonverbal presentation skills, which are assessed by two external experts from a human resources department. We extract multimodal features, such as spoken utterances, acoustic features, and the amount of body motion, to estimate the presentation skills. We propose two approaches, early fusing and late fusing, for the regression models based on multimodal instance weighting adaptation. The experimental results show that the early fusing regression model with instance weighting adaptation achieved (rho =0.39) for the Pearson correlation, which presents the regression accuracy for the clarity of presentation goal elements. In the maximum case, the accuracy (correlation coefficient) is improved from (-0.34) to +0.35 by instance weighting adaptation.","PeriodicalId":17529,"journal":{"name":"Journal on Multimodal User Interfaces","volume":"33 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2021-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Facial expression and action unit recognition augmented by their dependencies on graph convolutional networks 基于图卷积网络增强的面部表情和动作单元识别

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-26 DOI: 10.1007/s12193-020-00363-7

Jun He, Xiaocui Yu, Bo Sun, Lejun Yu

引用次数: 7

Neighborhood based decision theoretic rough set under dynamic granulation for BCI motor imagery classification 动态粒化下基于邻域决策理论粗糙集的脑机接口运动图像分类

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-25 DOI: 10.1007/s12193-020-00358-4

K. Renuga Devi, H. Hannah Inbarani

引用次数: 11

Comparing mind perception in strategic exchanges: human-agent negotiation, dictator and ultimatum games 战略交流中的心理感知比较:人-代理谈判、独裁者与最后通牒博弈

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-21 DOI: 10.1007/s12193-020-00356-6

Minha Lee, Gale M. Lucas, J. Gratch

引用次数: 13

A novel focus encoding scheme for addressee detection in multiparty interaction using machine learning algorithms 一种基于机器学习算法的多方交互中地址检测的焦点编码方案

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-17 DOI: 10.1007/s12193-020-00361-9

Usman Malik, Mukesh Barange, Julien Saunier, A. Pauchet

引用次数: 3

PLAAN: Pain Level Assessment with Anomaly-detection based Network PLAAN:基于异常检测网络的疼痛水平评估

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-06 DOI: 10.1007/s12193-020-00362-8

Yi Li, Shreya Ghosh, Jyoti Joshi

引用次数: 6

Internet-based tailored virtual human health intervention to promote colorectal cancer screening: design guidelines from two user studies 基于互联网的定制虚拟人类健康干预以促进结直肠癌筛查:来自两项用户研究的设计指南

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2021-01-02 DOI: 10.1007/s12193-020-00357-5

Mohan S Zalake, F. Tavassoli, Kyle A. Duke, T. George, François Modave, J. Neil, Janice L. Krieger, B. Lok

引用次数: 4

Words of encouragement: how praise delivered by a social robot changes children’s mindset for learning 鼓励的话语:社交机器人的赞美如何改变孩子的学习心态

IF 2.9 3区计算机科学

Journal on Multimodal User Interfaces Pub Date : 2020-11-24 DOI: 10.1007/s12193-020-00353-9

Daniel P. Davison, F. Wijnen, V. Charisi, J. van der Meij, D. Reidsma, V. Evers

引用次数: 14