Xiao Chen , Zhen Liu , Jiangjian Xiao , Tingting Liu , Yumeng Zhao
{"title":"基于情绪信息注意的依赖差分步态感知情绪","authors":"Xiao Chen , Zhen Liu , Jiangjian Xiao , Tingting Liu , Yumeng Zhao","doi":"10.1016/j.cogsys.2023.101150","DOIUrl":null,"url":null,"abstract":"<div><p>Perceiving human emotions is crucial in the realm of affective computing<span>. As a nonverbal biological feature, gait plays a significant role in this field, owing to its resistance to manipulation or replication. In this paper, we propose a gait-based emotion perception framework called Dependency-Difference Gait (DDG), which can extract emotional features from gait patterns comprehensively and efficiently. We also introduce a method of spatial–temporal difference representation, which constructs the static spatial difference information within frames and dynamic temporal difference information between frames. We abstract these details as difference information and fuse them with the dependency information extracted from the original sequence. Our approach not only breaks the limitations of hand-crafted features, but also enables the extraction of a broader spectrum of emotional features. Additionally, we present the Emotional Information Attention (EIA) mechanism, allowing DDG to focus on key joints and frames based on the quantity of emotional information. Experimental and visualization results substantiate the effectiveness of the DDG and EIA. In the quality analysis, we find that selecting a few number of joints with a substantial amount of emotional information is beneficial for emotion classification. However, selecting a few frames can disrupt the temporal structure of the sequence, resulting in suboptimal performance.</span></p></div>","PeriodicalId":55242,"journal":{"name":"Cognitive Systems Research","volume":null,"pages":null},"PeriodicalIF":2.1000,"publicationDate":"2023-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"DDG: Dependency-difference gait based on emotional information attention for perceiving emotions from gait\",\"authors\":\"Xiao Chen , Zhen Liu , Jiangjian Xiao , Tingting Liu , Yumeng Zhao\",\"doi\":\"10.1016/j.cogsys.2023.101150\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Perceiving human emotions is crucial in the realm of affective computing<span>. As a nonverbal biological feature, gait plays a significant role in this field, owing to its resistance to manipulation or replication. In this paper, we propose a gait-based emotion perception framework called Dependency-Difference Gait (DDG), which can extract emotional features from gait patterns comprehensively and efficiently. We also introduce a method of spatial–temporal difference representation, which constructs the static spatial difference information within frames and dynamic temporal difference information between frames. We abstract these details as difference information and fuse them with the dependency information extracted from the original sequence. Our approach not only breaks the limitations of hand-crafted features, but also enables the extraction of a broader spectrum of emotional features. Additionally, we present the Emotional Information Attention (EIA) mechanism, allowing DDG to focus on key joints and frames based on the quantity of emotional information. Experimental and visualization results substantiate the effectiveness of the DDG and EIA. In the quality analysis, we find that selecting a few number of joints with a substantial amount of emotional information is beneficial for emotion classification. However, selecting a few frames can disrupt the temporal structure of the sequence, resulting in suboptimal performance.</span></p></div>\",\"PeriodicalId\":55242,\"journal\":{\"name\":\"Cognitive Systems Research\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2023-08-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Cognitive Systems Research\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1389041723000785\",\"RegionNum\":3,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cognitive Systems Research","FirstCategoryId":"102","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1389041723000785","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
DDG: Dependency-difference gait based on emotional information attention for perceiving emotions from gait
Perceiving human emotions is crucial in the realm of affective computing. As a nonverbal biological feature, gait plays a significant role in this field, owing to its resistance to manipulation or replication. In this paper, we propose a gait-based emotion perception framework called Dependency-Difference Gait (DDG), which can extract emotional features from gait patterns comprehensively and efficiently. We also introduce a method of spatial–temporal difference representation, which constructs the static spatial difference information within frames and dynamic temporal difference information between frames. We abstract these details as difference information and fuse them with the dependency information extracted from the original sequence. Our approach not only breaks the limitations of hand-crafted features, but also enables the extraction of a broader spectrum of emotional features. Additionally, we present the Emotional Information Attention (EIA) mechanism, allowing DDG to focus on key joints and frames based on the quantity of emotional information. Experimental and visualization results substantiate the effectiveness of the DDG and EIA. In the quality analysis, we find that selecting a few number of joints with a substantial amount of emotional information is beneficial for emotion classification. However, selecting a few frames can disrupt the temporal structure of the sequence, resulting in suboptimal performance.
期刊介绍:
Cognitive Systems Research is dedicated to the study of human-level cognition. As such, it welcomes papers which advance the understanding, design and applications of cognitive and intelligent systems, both natural and artificial.
The journal brings together a broad community studying cognition in its many facets in vivo and in silico, across the developmental spectrum, focusing on individual capacities or on entire architectures. It aims to foster debate and integrate ideas, concepts, constructs, theories, models and techniques from across different disciplines and different perspectives on human-level cognition. The scope of interest includes the study of cognitive capacities and architectures - both brain-inspired and non-brain-inspired - and the application of cognitive systems to real-world problems as far as it offers insights relevant for the understanding of cognition.
Cognitive Systems Research therefore welcomes mature and cutting-edge research approaching cognition from a systems-oriented perspective, both theoretical and empirically-informed, in the form of original manuscripts, short communications, opinion articles, systematic reviews, and topical survey articles from the fields of Cognitive Science (including Philosophy of Cognitive Science), Artificial Intelligence/Computer Science, Cognitive Robotics, Developmental Science, Psychology, and Neuroscience and Neuromorphic Engineering. Empirical studies will be considered if they are supplemented by theoretical analyses and contributions to theory development and/or computational modelling studies.