Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games最新文献

筛选

英文中文

S2M-Net: Speech Driven Three-party Conversational Motion Synthesis Networks S2M-Net:语音驱动的三方会话运动合成网络

Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games Pub Date : 2022-11-03 DOI: 10.1145/3561975.3562954

Aobo Jin, Qixin Deng, Zhiwei Deng

{"title":"S2M-Net: Speech Driven Three-party Conversational Motion Synthesis Networks","authors":"Aobo Jin, Qixin Deng, Zhiwei Deng","doi":"10.1145/3561975.3562954","DOIUrl":"https://doi.org/10.1145/3561975.3562954","url":null,"abstract":"In this paper we propose a novel conditional generative adversarial network (cGAN) architecture, called S2M-Net, to holistically synthesize realistic three-party conversational animations based on acoustic speech input together with speaker marking (i.e., the speaking time of each interlocutor). Specifically, based on a pre-collected three-party conversational motion dataset, we design and train the S2M-Net for three-party conversational animation synthesis. In the architecture, a generator contains a LSTM encoder to encode a sequence of acoustic speech features to a latent vector that is further fed into a transform unit to transform the latent vector into a gesture kinematics space. Then, the output of this transform unit is fed into a LSTM decoder to generate corresponding three-party conversational gesture kinematics. Meanwhile, a discriminator is implemented to check whether an input sequence of three-party conversational gesture kinematics is real or fake. To evaluate our method, besides quantitative and qualitative evaluations, we also conducted paired comparison user studies to compare it with the state of the art.","PeriodicalId":246179,"journal":{"name":"Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games","volume":"188 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132389355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

首页上一页