Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge最新文献

筛选
英文 中文
Multimodal Measurement of Depression Using Deep Learning Models 使用深度学习模型的多模态抑郁测量
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133948
Le Yang, D. Jiang, Xiaohan Xia, Ercheng Pei, Meshia Cédric Oveneke, H. Sahli
{"title":"Multimodal Measurement of Depression Using Deep Learning Models","authors":"Le Yang, D. Jiang, Xiaohan Xia, Ercheng Pei, Meshia Cédric Oveneke, H. Sahli","doi":"10.1145/3133944.3133948","DOIUrl":"https://doi.org/10.1145/3133944.3133948","url":null,"abstract":"This paper addresses multi-modal depression analysis. We propose a multi-modal fusion framework composed of deep convolutional neural network (DCNN) and deep neural network (DNN) models. Our framework considers audio, video and text streams. For each modality, handcrafted feature descriptors are input into a DCNN to learn high-level global features with compact dynamic information, then the learned features are fed to a DNN to predict the PHQ-8 scores. For multi-modal fusion, the estimated PHQ-8 scores from the three modalities are integrated in a DNN to obtain the final PHQ-8 score. Moreover, in this work, we propose new feature descriptors for text and video. For the text descriptors, we select the participant»s answers to the questions associated with psychoanalytic aspects of depression, such as sleep disorder, and make use of the Paragraph Vector (PV) to learn the distributed representations of these sentences. For the video descriptors, we propose a new global descriptor, the Histogram of Displacement Range (HDR), calculated directly from the facial landmarks to measure their displacements and speed. Experiments have been carried out on the AVEC2017 depression sub-challenge dataset. The obtained results show that the proposed depression recognition framework obtains very promising accuracy, with the root mean square error (RMSE) as 4.653, mean absolute error (MAE) as 3.980 on the development set, and RMSE as 5.974, MAE as 5.163 on the test set.","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"344 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123104085","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 118
Topic Modeling Based Multi-modal Depression Detection 基于主题建模的多模态抑郁检测
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133945
Yuan Gong, C. Poellabauer
{"title":"Topic Modeling Based Multi-modal Depression Detection","authors":"Yuan Gong, C. Poellabauer","doi":"10.1145/3133944.3133945","DOIUrl":"https://doi.org/10.1145/3133944.3133945","url":null,"abstract":"Major depressive disorder is a common mental disorder that affects almost 7% of the adult U.S. population. The 2017 Audio/Visual Emotion Challenge (AVEC) asks participants to build a model to predict depression levels based on the audio, video, and text of an interview ranging between 7-33 minutes. Since averaging features over the entire interview will lose most temporal information, how to discover, capture, and preserve useful temporal details for such a long interview are significant challenges. Therefore, we propose a novel topic modeling based approach to perform context-aware analysis of the recording. Our experiments show that the proposed approach outperforms context-unaware methods and the challenge baselines for all metrics.","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122795805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 91
Body Language Without a Body: Nonverbal Communication in Technology Mediated Settings 没有身体的肢体语言:技术介导下的非语言交际
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133954
A. Vinciarelli
{"title":"Body Language Without a Body: Nonverbal Communication in Technology Mediated Settings","authors":"A. Vinciarelli","doi":"10.1145/3133944.3133954","DOIUrl":"https://doi.org/10.1145/3133944.3133954","url":null,"abstract":"Humans are wired for face-to-face interaction because this was the only possible and available setting during the long evolutionary process that has led to Homo Sapiens. At the moment an increasingly significant fraction of our interactions take place in technology mediated settings, it is important to investigate how such a wiring - mainly corresponding to neural processes - reacts and adapts to them. This talk focuses in particular on how nonverbal communication - one of the main channels through which people convey socially and psychologically relevant information - plays a role in settings where natural nonverbal cues (facial expressions, vocalizations, gestures, etc.) are no longer available. Such an issue is important not only from a technological point of view (it can help to design interaction and communication technologies that better address human needs), but also from a societal one (it can help to understand major phenomena such as cyberbullyism and virality).","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127619102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Session details: Keynote: Pr. Alessandro Vinciarelli
F. Ringeval
{"title":"Session details: Keynote: Pr. Alessandro Vinciarelli","authors":"F. Ringeval","doi":"10.1145/3252891","DOIUrl":"https://doi.org/10.1145/3252891","url":null,"abstract":"","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114546578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge 第七届视听情感挑战研讨会论文集
F. Ringeval, Björn Schuller, M. Valstar, J. Gratch, R. Cowie, M. Pantic
{"title":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","authors":"F. Ringeval, Björn Schuller, M. Valstar, J. Gratch, R. Cowie, M. Pantic","doi":"10.1145/3133944","DOIUrl":"https://doi.org/10.1145/3133944","url":null,"abstract":"It is our great pleasure to welcome you to the 7th Audio-Visual Emotion Challenge -- AVEC'17, held in conjunction with the ACM Multimedia 2017 in Mountain View, CA, USA. \u0000 \u0000This year's challenge and associated workshop continues to push the boundaries of audio-visual emotion and depression recognition towards real-life applications of behavioural computing. Looking back in the history of AVEC, the first challenge posed the problem of detecting discrete emotion classes on a large set of natural behaviour data. The second AVEC extended this problem to the prediction of continuous valued dimensional affect. This problem was enlarged further for the third edition to include the prediction of self-reported severity of depression. The fourth edition was a refined run with enriched annotations. The fifth AVEC introduced physiological signals, along with audio-visual data, for the prediction of dimensional affect. In the sixth edition, we introduced human-agent interactions for depression analysis, in addition to affect recognition. Finally, this year we've focused the study of affect from human behaviours captured 'in-the-wild', along with depression analysis from human-agent interactions. \u0000 \u0000The mission of AVEC challenge and workshop series is to provide a common benchmark test set for individual multimodal information processing and to bring together the audio, video and audio-visual emotion recognition communities, to compare the relative merits of the approaches to emotion recognition under well-defined and strictly comparable conditions and establish to what extent fusion of the approaches is possible and beneficial. The main underlying motivation is the need to advance emotion recognition and depression estimation for multimedia retrieval to a level where behaviours can be reliably sensed in real-life conditions, as this is exactly the type of data that applications would have to face in the real world.","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122813900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
Hybrid Depression Classification and Estimation from Audio Video and Text Information 基于音视频和文字信息的混合抑郁分类与估计
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133950
Le Yang, H. Sahli, Xiaohan Xia, Ercheng Pei, Meshia Cédric Oveneke, D. Jiang
{"title":"Hybrid Depression Classification and Estimation from Audio Video and Text Information","authors":"Le Yang, H. Sahli, Xiaohan Xia, Ercheng Pei, Meshia Cédric Oveneke, D. Jiang","doi":"10.1145/3133944.3133950","DOIUrl":"https://doi.org/10.1145/3133944.3133950","url":null,"abstract":"In this paper, we design a hybrid depression classification and depression estimation framework from audio, video and text descriptors. It contains three main components: 1) Deep Convolutional Neural Network (DCNN) and Deep Neural Network (DNN) based audio visual multi-modal depression recognition frameworks, trained with depressed and not-depressed participants, respectively; 2) Paragraph Vector (PV), Support Vector Machine (SVM) and Random Forest based depression classification framework from the interview transcripts; 3) A multivariate regression model fusing the audio visual PHQ-8 estimations from the depressed and not-depressed DCNN-DNN models, and the depression classification result from the text information. In the DCNN-DNN based depression estimation framework, audio/video feature descriptors are first input into a DCNN to learn high-level features, which are then fed to a DNN to predict the PHQ-8 score. Initial predictions from the two modalities are fused via a DNN model. In the PV-SVM and Random Forest based depression classification framework, we explore semantic-related text features using PV, as well as global text-features. Experiments have been carried out on the Distress Analysis Interview Corpus - Wizard of Oz (DAIC-WOZ) dataset for the Depression Sub-challenge at the 2017 Audio-Visual Emotion Challenge (AVEC), results show that the proposed depression recognition framework obtains very promising results, with root mean square error (RMSE) as 3.088, mean absolute error (MAE) as 2.477 on the development set, and RMSE as 5.400, MAE as 4.359 on the test set, which are all lower than the baseline results.","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124091945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 65
Session details: AVEC 2017 Part 2 会议详情:AVEC 2017第二部分
Jing Han
{"title":"Session details: AVEC 2017 Part 2","authors":"Jing Han","doi":"10.1145/3252894","DOIUrl":"https://doi.org/10.1145/3252894","url":null,"abstract":"","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123550409","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
AVEC 2017: Real-life Depression, and Affect Recognition Workshop and Challenge AVEC 2017:现实生活中的抑郁症和情感识别研讨会和挑战
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133953
F. Ringeval, Björn W. Schuller, M. Valstar, Jonathan Gratch, R. Cowie, Stefan Scherer, Sharon Mozgai, Nicholas Cummins, Maximilian Schmitt, Maja Pantic
{"title":"AVEC 2017: Real-life Depression, and Affect Recognition Workshop and Challenge","authors":"F. Ringeval, Björn W. Schuller, M. Valstar, Jonathan Gratch, R. Cowie, Stefan Scherer, Sharon Mozgai, Nicholas Cummins, Maximilian Schmitt, Maja Pantic","doi":"10.1145/3133944.3133953","DOIUrl":"https://doi.org/10.1145/3133944.3133953","url":null,"abstract":"The Audio/Visual Emotion Challenge and Workshop (AVEC 2017) \"Real-life depression, and affect\" will be the seventh competition event aimed at comparison of multimedia processing and machine learning methods for automatic audiovisual depression and emotion analysis, with all participants competing under strictly the same conditions. The goal of the Challenge is to provide a common benchmark test set for multimodal information processing and to bring together the depression and emotion recognition communities, as well as the audiovisual processing communities, to compare the relative merits of the various approaches to depression and emotion recognition from real-life data. This paper presents the novelties introduced this year, the challenge guidelines, the data used, and the performance of the baseline system on the two proposed tasks: dimensional emotion recognition (time and value-continuous), and dimensional depression estimation (value-continuous).","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"64 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126672746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 17
Depression Severity Prediction Based on Biomarkers of Psychomotor Retardation 基于精神运动迟缓生物标志物的抑郁严重程度预测
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133947
Zafi Sherhan Syed, K. Sidorov, David Marshall
{"title":"Depression Severity Prediction Based on Biomarkers of Psychomotor Retardation","authors":"Zafi Sherhan Syed, K. Sidorov, David Marshall","doi":"10.1145/3133944.3133947","DOIUrl":"https://doi.org/10.1145/3133944.3133947","url":null,"abstract":"This paper addresses the AVEC 2017 ? Depression Sub-Challenge, where the objective is to propose methods which can aid automated prediction of depression severity. In this paper, we specifically focus on biomarkers of psychomotor retardation, which are a key trait of depressive episodes, to propose three sets of methods. We propose a novel set of temporal features (which we called \"turbulence features\") and show their effectiveness. We offer a novel methodology to target specific craniofacial movements indicative of psychomotor retardation and hence of depression. Further, we present a novel method for quantifying abnormalities of speech spectra of individuals with depression using Fisher vector encoding of spectral low level descriptors (LLDs). So far, in the AVEC challenge on prediction of patient health questionnaire (PHQ) scores on the Test set, we achieve a root mean square error (RMSE) score of 6.34 and a mean absolute error (MAE) score of 5.30, both of which are better than the best results on the AVEC test set as given in the baseline paper i.e. 6.97 and 5.66, respectively. This suggests that our method is a viable proof of concept and may lead to fully automated objective depression screening protocols.","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124411126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 35
Multimodal Multi-task Learning for Dimensional and Continuous Emotion Recognition 多维连续情绪识别的多模态多任务学习
Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge Pub Date : 2017-10-23 DOI: 10.1145/3133944.3133949
Shizhe Chen, Qin Jin, Jinming Zhao, Shuai Wang
{"title":"Multimodal Multi-task Learning for Dimensional and Continuous Emotion Recognition","authors":"Shizhe Chen, Qin Jin, Jinming Zhao, Shuai Wang","doi":"10.1145/3133944.3133949","DOIUrl":"https://doi.org/10.1145/3133944.3133949","url":null,"abstract":"Automatic emotion recognition is a challenging task which can make great impact on improving natural human computer interactions. In this paper, we present our effort for the Affect Subtask in the Audio/Visual Emotion Challenge (AVEC) 2017, which requires participants to perform continuous emotion prediction on three affective dimensions: Arousal, Valence and Likability based on the audiovisual signals. We highlight three aspects of our solutions: 1) we explore and fuse different hand-crafted and deep learned features from all available modalities including acoustic, visual, and textual modalities, and we further consider the interlocutor influence for the acoustic features; 2) we compare the effectiveness of non-temporal model SVR and temporal model LSTM-RNN and show that the LSTM-RNN can not only alleviate the feature engineering efforts such as construction of contextual features and feature delay, but also improve the recognition performance significantly; 3) we apply multi-task learning strategy for collaborative prediction of multiple emotion dimensions with shared representations according to the fact that different emotion dimensions are correlated with each other. Our solutions achieve the CCC of 0.675, 0.756 and 0.509 on arousal, valence, and likability respectively on the challenge testing set, which outperforms the baseline system with corresponding CCC of 0.375, 0.466, and 0.246 on arousal, valence, and likability.","PeriodicalId":412897,"journal":{"name":"Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132288904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 114
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书