IEEE Transactions on Affective Computing最新文献

Individual-Aware Attention Modulation for Unseen Speaker Emotion Recognition 个体感知注意力调制用于识别看不见的说话者情绪

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2024-11-15 DOI: 10.1109/taffc.2024.3498937

Yuanbo Fang, Xiaofen Xing, Zhaojie Chu, Yifeng Du, Xiangmin Xu

引用次数: 0

Sparse Emotion Dictionary and CWT Spectrogram Fusion with Multi-head Self-Attention for Depression Recognition in Parkinson's Disease Patients 稀疏情绪字典和 CWT 频谱图与多头自我注意力融合，用于帕金森病患者的抑郁识别

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2024-11-14 DOI: 10.1109/taffc.2024.3498009

Jian Li, Yuliang Zhao, Yinghao Liu, Huawei Zhang, Peng Shan, Yuanyi Wu, Wanyue Wang, Yulin Wang

引用次数: 0

EEG-Based Cross-Subject Emotion Recognition Using Sparse Bayesian Learning with Enhanced Covariance Alignment 利用稀疏贝叶斯学习与增强协方差对齐进行基于脑电图的跨受试者情绪识别

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2024-11-14 DOI: 10.1109/taffc.2024.3497897

Wenlong Wang, Feifei Qi, Weichen Huang, Yuanqing Li, Zhuliang Yu, Wei Wu

引用次数: 0

A Low-Rank Matching Attention Based Cross-Modal Feature Fusion Method for Conversational Emotion Recognition 基于低库匹配注意力的对话情绪识别跨模态特征融合方法

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2024-11-14 DOI: 10.1109/taffc.2024.3498443

Yuntao Shou, Huan Liu, Xiangyong Cao, Deyu Meng, Bo Dong

引用次数: 0

Exploring Complexity of Facial Dynamics in Autism Spectrum Disorder 探索自闭症谱系障碍面部动力学的复杂性

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2021-09-20 DOI: 10.1109/TAFFC.2021.3113876

Pradeep Raj Krishnappa Babu;J. Matias Di Martino;Zhuoqing Chang;Sam Perochon;Kimberly L. H. Carpenter;Scott Compton;Steven Espinosa;Geraldine Dawson;Guillermo Sapiro

{"title":"Exploring Complexity of Facial Dynamics in Autism Spectrum Disorder","authors":"Pradeep Raj Krishnappa Babu;J. Matias Di Martino;Zhuoqing Chang;Sam Perochon;Kimberly L. H. Carpenter;Scott Compton;Steven Espinosa;Geraldine Dawson;Guillermo Sapiro","doi":"10.1109/TAFFC.2021.3113876","DOIUrl":"10.1109/TAFFC.2021.3113876","url":null,"abstract":"Atypical facial expression is one of the early symptoms of autism spectrum disorder (ASD) characterized by reduced regularity and lack of coordination of facial movements. Automatic quantification of these behaviors can offer novel biomarkers for screening, diagnosis, and treatment monitoring of ASD. In this work, 40 toddlers with ASD and 396 typically developing toddlers were shown developmentally-appropriate and engaging movies presented on a smart tablet during a well-child pediatric visit. The movies consisted of social and non-social dynamic scenes designed to evoke certain behavioral and affective responses. The front-facing camera of the tablet was used to capture the toddlers’ face. Facial landmarks’ dynamics were then automatically computed using computer vision algorithms. Subsequently, the complexity of the landmarks’ dynamics was estimated for the eyebrows and mouth regions using multiscale entropy. Compared to typically developing toddlers, toddlers with ASD showed higher complexity (i.e., less predictability) in these landmarks’ dynamics. This complexity in facial dynamics contained novel information not captured by traditional facial affect analyses. These results suggest that computer vision analysis of facial landmark movements is a promising approach for detecting and quantifying early behavioral symptoms associated with ASD.","PeriodicalId":13131,"journal":{"name":"IEEE Transactions on Affective Computing","volume":"14 2","pages":"919-930"},"PeriodicalIF":11.2,"publicationDate":"2021-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9541259","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9581191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Indirect Identification of Perinatal Psychosocial Risks From Natural Language 从自然语言间接识别围产期心理社会风险

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2021-03-11 DOI: 10.1109/TAFFC.2021.3079282

Kristen C. Allen;Alex Davis;Tamar Krishnamurti

{"title":"Indirect Identification of Perinatal Psychosocial Risks From Natural Language","authors":"Kristen C. Allen;Alex Davis;Tamar Krishnamurti","doi":"10.1109/TAFFC.2021.3079282","DOIUrl":"10.1109/TAFFC.2021.3079282","url":null,"abstract":"During the perinatal period, psychosocial health risks, including depression and intimate partner violence, are associated with serious adverse health outcomes for birth parents and children. To appropriately intervene, healthcare professionals must first identify those at risk, yet stigma often prevents people from directly disclosing the information needed to prompt an assessment. In this research we use short diary entries to indirectly elicit information that could indicate psychosocial risks, then examine patterns that emerge in the language of those at risk. We find that diary entries exhibit consistent themes, extracted using topic modeling, and emotional perspective, drawn from dictionary-informed sentiment features. Using these features, we use regularized regression to predict screening measures for depression and psychological aggression by an intimate partner. Journal text entries quantified through topic models and sentiment features show promise for depression prediction, corresponding with self-reported screening measures almost as well as closed-form questions. Text-based features are less useful in predicting intimate partner violence, but topic models generate themes that align with known risk correlates. The indirect features uncovered in this research could aid in the detection and analysis of stigmatized risks.","PeriodicalId":13131,"journal":{"name":"IEEE Transactions on Affective Computing","volume":"14 2","pages":"1506-1519"},"PeriodicalIF":11.2,"publicationDate":"2021-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAFFC.2021.3079282","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9928984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Interpretation of Depression Detection Models via Feature Selection Methods 用特征选择方法解释抑郁症检测模型

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2020-11-10 DOI: 10.1109/TAFFC.2020.3035535

Sharifa Alghowinem;Tom Gedeon;Roland Goecke;Jeffrey F. Cohn;Gordon Parker

{"title":"Interpretation of Depression Detection Models via Feature Selection Methods","authors":"Sharifa Alghowinem;Tom Gedeon;Roland Goecke;Jeffrey F. Cohn;Gordon Parker","doi":"10.1109/TAFFC.2020.3035535","DOIUrl":"10.1109/TAFFC.2020.3035535","url":null,"abstract":"Given the prevalence of depression worldwide and its major impact on society, several studies employed artificial intelligence modelling to automatically detect and assess depression. However, interpretation of these models and cues are rarely discussed in detail in the AI community, but have received increased attention lately. In this article, we aim to analyse the commonly selected features using a proposed framework of several feature selection methods and their effect on the classification results, which will provide an interpretation of the depression detection model. The developed framework aggregates and selects the most promising features for modelling depression detection from 38 feature selection algorithms of different categories. Using three real-world depression datasets, 902 behavioural cues were extracted from speech behaviour, speech prosody, eye movement and head pose. To verify the generalisability of the proposed framework, we applied the entire process to depression datasets individually and when combined. The results from the proposed framework showed that speech behaviour features (e.g. pauses) are the most distinctive features of the depression detection model. From the speech prosody modality, the strongest feature groups were F0, HNR, formants, and MFCC, while for the eye activity modality they were left-right eye movement and gaze direction, and for the head modality it was yaw head movement. Modelling depression detection using the selected features (even though there are only 9 features) outperformed using all features in all the individual and combined datasets. Our feature selection framework did not only provide an interpretation of the model, but was also able to produce a higher accuracy of depression detection with a small number of features in varied datasets. This could help to reduce the processing time needed to extract features and creating the model.","PeriodicalId":13131,"journal":{"name":"IEEE Transactions on Affective Computing","volume":"14 1","pages":"133-152"},"PeriodicalIF":11.2,"publicationDate":"2020-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAFFC.2020.3035535","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9155516","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 33

Multi-Label Multi-Task Deep Learning for Behavioral Coding 行为编码的多标签多任务深度学习

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2019-11-08 DOI: 10.1109/TAFFC.2019.2952113

James Gibson;David C. Atkins;Torrey A. Creed;Zac Imel;Panayiotis Georgiou;Shrikanth Narayanan

{"title":"Multi-Label Multi-Task Deep Learning for Behavioral Coding","authors":"James Gibson;David C. Atkins;Torrey A. Creed;Zac Imel;Panayiotis Georgiou;Shrikanth Narayanan","doi":"10.1109/TAFFC.2019.2952113","DOIUrl":"10.1109/TAFFC.2019.2952113","url":null,"abstract":"We propose a methodology for estimating human behaviors in psychotherapy sessions using multi-label and multi-task learning paradigms. We discuss the problem of behavioral coding in which data of human interactions are annotated with labels to describe relevant human behaviors of interest. We describe two related, yet distinct, corpora consisting of therapist-client interactions in psychotherapy sessions. We experimentally compare the proposed learning approaches for estimating behaviors of interest in these datasets. Specifically, we compare single and multiple label learning approaches, single and multiple task learning approaches, and evaluate the performance of these approaches when incorporating turn context. We demonstrate that the best multi-label, multi-task learning model with turn context achieves 18.9 and 19.5 percent absolute improvements with respect to a logistic regression classifier (for each behavioral coding task respectively) and 6.4 and 6.1 percent absolute improvements with respect to the best single-label, single-task deep neural network models. Lastly, we discuss the insights these modeling paradigms provide into these complex interactions including key commonalities and differences of behaviors within and between the two prevalent psychotherapy approaches–Motivational Interviewing and Cognitive Behavioral Therapy–considered.","PeriodicalId":13131,"journal":{"name":"IEEE Transactions on Affective Computing","volume":"13 1","pages":"508-518"},"PeriodicalIF":11.2,"publicationDate":"2019-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAFFC.2019.2952113","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10588865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 24

Applying Probabilistic Programming to Affective Computing 概率规划在情感计算中的应用

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2019-03-15 DOI: 10.1109/TAFFC.2019.2905211

Desmond C. Ong;Harold Soh;Jamil Zaki;Noah D. Goodman

{"title":"Applying Probabilistic Programming to Affective Computing","authors":"Desmond C. Ong;Harold Soh;Jamil Zaki;Noah D. Goodman","doi":"10.1109/TAFFC.2019.2905211","DOIUrl":"10.1109/TAFFC.2019.2905211","url":null,"abstract":"Affective Computing is a rapidly growing field spurred by advancements in artificial intelligence, but often, held back by the inability to translate psychological theories of emotion into tractable computational models. To address this, we propose a probabilistic programming approach to affective computing, which models psychological-grounded theories as generative models of emotion, and implements them as stochastic, executable computer programs. We first review probabilistic approaches that integrate reasoning about emotions with reasoning about other latent mental states (e.g., beliefs, desires) in context. Recently-developed probabilistic programming languages offer several key desidarata over previous approaches, such as: (i) flexibility in representing emotions and emotional processes; (ii) modularity and compositionality; (iii) integration with deep learning libraries that facilitate efficient inference and learning from large, naturalistic data; and (iv) ease of adoption. Furthermore, using a probabilistic programming framework allows a standardized platform for theory-building and experimentation: Competing theories (e.g., of appraisal or other emotional processes) can be easily compared via modular substitution of code followed by model comparison. To jumpstart adoption, we illustrate our points with executable code that researchers can easily modify for their own models. We end with a discussion of applications and future directions of the probabilistic programming approach.","PeriodicalId":13131,"journal":{"name":"IEEE Transactions on Affective Computing","volume":"12 2","pages":"306-317"},"PeriodicalIF":11.2,"publicationDate":"2019-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAFFC.2019.2905211","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39034946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health 预测明天情绪、压力和健康的个性化多任务学习

IF 11.2 2区计算机科学

IEEE Transactions on Affective Computing Pub Date : 2017-12-19 DOI: 10.1109/TAFFC.2017.2784832

Sara Taylor;Natasha Jaques;Ehimwenma Nosakhare;Akane Sano;Rosalind Picard

{"title":"Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health","authors":"Sara Taylor;Natasha Jaques;Ehimwenma Nosakhare;Akane Sano;Rosalind Picard","doi":"10.1109/TAFFC.2017.2784832","DOIUrl":"10.1109/TAFFC.2017.2784832","url":null,"abstract":"While accurately predicting mood and wellbeing could have a number of important clinical benefits, traditional machine learning (ML) methods frequently yield low performance in this domain. We posit that this is because a one-size-fits-all machine learning model is inherently ill-suited to predicting outcomes like mood and stress, which vary greatly due to individual differences. Therefore, we employ Multitask Learning (MTL) techniques to train personalized ML models which are customized to the needs of each individual, but still leverage data from across the population. Three formulations of MTL are compared: i) MTL deep neural networks, which share several hidden layers but have final layers unique to each task; ii) Multi-task Multi-Kernel learning, which feeds information across tasks through kernel weights on feature types; and iii) a Hierarchical Bayesian model in which tasks share a common Dirichlet Process prior. We offer the code for this work in open source. These techniques are investigated in the context of predicting future mood, stress, and health using data collected from surveys, wearable sensors, smartphone logs, and the weather. Empirical results demonstrate that using MTL to account for individual differences provides large performance improvements over traditional machine learning methods and provides personalized, actionable insights.","PeriodicalId":13131,"journal":{"name":"IEEE Transactions on Affective Computing","volume":"11 2","pages":"200-213"},"PeriodicalIF":11.2,"publicationDate":"2017-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1109/TAFFC.2017.2784832","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"38006121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 151