Companion Publication of the 2022 International Conference on Multimodal Interaction最新文献_第3页

Enabling Non-Technical Domain Experts to Create Robot-Assisted Therapeutic Scenarios via Visual Programming 使非技术领域的专家能够通过可视化编程创建机器人辅助的治疗方案

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558072

Christian Schütze, André Groß, B. Wrede, Birte Richter

引用次数: 3

Exploring Facial Metric Normalization For Within- and Between-Subject Comparisons in a Multimodal Health Monitoring Agent 在多模态健康监测代理中探索受试者内部和受试者之间比较的面部度量归一化

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558071

Oliver Roesler, Hardik Kothare, William Burke, Michael Neumann, J. Liscombe, Andrew Cornish, Doug Habberstad, D. Pautler, David Suendermann-Oeft, Vikram Ramanarayanan

{"title":"Exploring Facial Metric Normalization For Within- and Between-Subject Comparisons in a Multimodal Health Monitoring Agent","authors":"Oliver Roesler, Hardik Kothare, William Burke, Michael Neumann, J. Liscombe, Andrew Cornish, Doug Habberstad, D. Pautler, David Suendermann-Oeft, Vikram Ramanarayanan","doi":"10.1145/3536220.3558071","DOIUrl":"https://doi.org/10.1145/3536220.3558071","url":null,"abstract":"The use of facial metrics obtained through remote web-based platforms has shown promising results for at-home assessment of facial function in multiple neurological and mental disorders. However, an important factor influencing the utility of the obtained metrics is the variability within and across participant sessions due to position and movement of the head relative to the camera. In this paper, we investigate two different facial landmark predictors in combination with four different normalization methods with respect to their effect on the utility of facial metrics obtained through a multimodal assessment platform. We analyzed 38 people with Parkinson’s disease (pPD) and 22 healthy controls who were asked to complete four interactive sessions, a week apart from each other. We find that metrics extracted through MediaPipe clearly outperform metrics extracted through OpenCV and Dlib in terms of test-retest reliability and patient-control discriminability. Furthermore, our results suggest that using the inter-caruncular distance to normalize all raw visual measurements prior to metric computation is optimal for between-subject analyses, while raw measurements (without normalization) can also be used for within-subject comparisons.","PeriodicalId":186796,"journal":{"name":"Companion Publication of the 2022 International Conference on Multimodal Interaction","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126456515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

ReCell: replicating recurrent cell for auto-regressive pose generation ReCell:复制复发细胞以产生自退姿态

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558801

V. Korzun, Anna Beloborodova, Arkady Ilin

引用次数: 5

How can Interaction Data be Contextualized with Mobile Sensing to Enhance Learning Engagement Assessment in Distance Learning? 如何将互动数据与移动传感结合起来，以加强远程学习的学习投入评估?

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558037

George-Petru Ciordas-Hertel, Daniel Biedermann, M. Winter, Julia Mordel, H. Drachsler

{"title":"How can Interaction Data be Contextualized with Mobile Sensing to Enhance Learning Engagement Assessment in Distance Learning?","authors":"George-Petru Ciordas-Hertel, Daniel Biedermann, M. Winter, Julia Mordel, H. Drachsler","doi":"10.1145/3536220.3558037","DOIUrl":"https://doi.org/10.1145/3536220.3558037","url":null,"abstract":"Multimodal learning analytics can enrich interaction data with contextual information through mobile sensing. Information about, for example, the physical environment, movement, physiological signals, or smart wearable usage. Through the use of smart wearables, contextual information can thus be captured and made available again to students in further processing steps so that they can reflect and annotate it. This paper describes a software infrastructure and a study design that successfully captured contextual information utilizing mobile sensing using students’ smart wearables in distance learning. In the conducted study, data was collected from the smartphones of 76 students as they self-directedly participated in an online learning unit using a learning management system (LMS) over a two-week period. During the students’ active phases in the LMS, interaction data as well as state and trait measurements were collected by the LMS. Simultaneously, hardware sensor data, app usage data, interaction with notifications, and ecological momentary assessments (EMA) were automatically but transparently collected from the students’ smartphones. Finally, this paper describes some preliminary insights from the study process and their implications for further data processing.","PeriodicalId":186796,"journal":{"name":"Companion Publication of the 2022 International Conference on Multimodal Interaction","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121851671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Predicting evaluations of entrepreneurial pitches based on multimodal nonverbal behavioral cues and self-reported characteristics 基于多模态非语言行为线索和自我报告特征的创业推介预测评价

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558041

Kostas Stoitsas, Itır Önal Ertuğrul, Werner Liebregts, Merel M. Jung

引用次数: 5

Head Movement Patterns during Face-to-Face Conversations Vary with Age 面对面交谈时的头部运动模式随年龄而变化

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3563366

Denisa Qori Mcdonald, C. Zampella, E. Sariyanidi, Aashvi Manakiwala, Ellis Dejardin, J. Herrington, R. Schultz, B. Tunç

{"title":"Head Movement Patterns during Face-to-Face Conversations Vary with Age","authors":"Denisa Qori Mcdonald, C. Zampella, E. Sariyanidi, Aashvi Manakiwala, Ellis Dejardin, J. Herrington, R. Schultz, B. Tunç","doi":"10.1145/3536220.3563366","DOIUrl":"https://doi.org/10.1145/3536220.3563366","url":null,"abstract":"Advances in computational behavior analysis have the potential to increase our understanding of behavioral patterns and developmental trajectories in neurotypical individuals, as well as in individuals with mental health conditions marked by motor, social, and emotional difficulties. This study focuses on investigating how head movement patterns during face–to–face conversations vary with age from childhood through adulthood. We rely on computer vision techniques due to their suitability for analysis of social behaviors in naturalistic settings, since video data capture can be unobtrusively embedded within conversations between two social partners. The methods in this work include unsupervised learning for movement pattern clustering, and supervised classification and regression as a function of age. The results demonstrate that 3–minute video recordings of head movements during conversations show patterns that distinguish between participants that are younger vs. older than 12 years with accuracy. Additionally, we extract relevant patterns of head movement upon which the age distinction was determined by our models.","PeriodicalId":186796,"journal":{"name":"Companion Publication of the 2022 International Conference on Multimodal Interaction","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128202449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Automatic facial expressions, gaze direction and head movements generation of a virtual agent 自动面部表情，凝视方向和头部运动生成的虚拟代理

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558806

Alice Delbosc, M. Ochs, S. Ayache

{"title":"Automatic facial expressions, gaze direction and head movements generation of a virtual agent","authors":"Alice Delbosc, M. Ochs, S. Ayache","doi":"10.1145/3536220.3558806","DOIUrl":"https://doi.org/10.1145/3536220.3558806","url":null,"abstract":"In this article, we present two models to jointly and automatically generate the head, facial and gaze movements of a virtual agent from acoustic speech features. Two architectures are explored: a Generative Adversarial Network and an Adversarial Encoder-Decoder. Head movements and gaze orientation are generated as 3D coordinates, while facial expressions are generated using action units based on the facial action coding system. A large corpus of almost 4 hours of videos, involving 89 different speakers is used to train our models. We extract the speech and visual features automatically from these videos using existing tools. The evaluation of these models is conducted objectively with measures such as density evaluation and a visualisation from PCA reduction, as well as subjectively through a users perceptive study. Our proposed methodology shows that on 15 seconds sequences, encoder-decoder architecture drastically improves the perception of generated behaviours in two criteria: the coordination with speech and the naturalness. Our code can be found in : https://github.com/aldelb/non-verbal-behaviours-generation.","PeriodicalId":186796,"journal":{"name":"Companion Publication of the 2022 International Conference on Multimodal Interaction","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128634927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Exploring the Benefits of Spatialized Multimodal Psychophysiological Insights for User Experience Research 探索空间化多模态心理生理学洞察对用户体验研究的好处

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3558039

F. Simard, Tomy Aumont, Sayeed A. D. Kizuk, Pascal E. Fortin

引用次数: 1

Training Computational Models of Group Processes without Groundtruth: the Self- vs External Assessment’s Dilemma 无基础真理的群体过程训练计算模型:自我评估与外部评估的困境

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-11-07 DOI: 10.1145/3536220.3563687

Lucien Maman, G. Volpe, G. Varni

{"title":"Training Computational Models of Group Processes without Groundtruth: the Self- vs External Assessment’s Dilemma","authors":"Lucien Maman, G. Volpe, G. Varni","doi":"10.1145/3536220.3563687","DOIUrl":"https://doi.org/10.1145/3536220.3563687","url":null,"abstract":"Supervised learning relies on the availability and reliability of the labels used to train computational models. In research areas such as Affective Computing and Social Signal Processing, such labels are usually extracted from multiple self- and/or external assessments. Labels are, then, either aggregated to produce a single groundtruth label, or all used during training, potentially resulting in degrading performance of the models. Defining a “true” label is, however, complex. Labels can be gathered at different times, with different tools, and may contain biases. Furthermore, multiple assessments are usually available for a same sample with potential contradictions. Thus, it is crucial to devise strategies that can take advantage of both self- and external assessments to train computational models without a reliable groundtruth. In this study, we designed and tested 3 of such strategies with the aim of mitigating the biases and making the models more robust to uncertain labels. Results show that the strategy based on weighting the loss during training according to a measure of disagreement improved the performances of the baseline, hence, underlining the potential of such an approach.","PeriodicalId":186796,"journal":{"name":"Companion Publication of the 2022 International Conference on Multimodal Interaction","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131049782","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion 改善就是改变:通过学习情绪变化来改善情绪预测

Companion Publication of the 2022 International Conference on Multimodal Interaction Pub Date : 2022-10-03 DOI: 10.1145/3536220.3563685

S. Narayana, Ramanathan Subramanian, Ibrahim Radwan, Roland Goecke

引用次数: 3