Chinese personalised text-to-speech synthesis for robot human–machine interaction

IF 1.5 Q3 AUTOMATION & CONTROL SYSTEMS
Bao Pang, Jun Teng, Qingyang Xu, Yong Song, Xianfeng Yuan, Yibin Li
{"title":"Chinese personalised text-to-speech synthesis for robot human–machine interaction","authors":"Bao Pang,&nbsp;Jun Teng,&nbsp;Qingyang Xu,&nbsp;Yong Song,&nbsp;Xianfeng Yuan,&nbsp;Yibin Li","doi":"10.1049/csy2.12098","DOIUrl":null,"url":null,"abstract":"<p>Speech interaction is an important means of robot interaction. With the rapid development of deep learning, end-to-end speech synthesis methods based on this technique have gradually become mainstream. Chinese deep learning-based speech synthesis techniques suffer from problems such as unstable synthesised speech, poor naturalness and poor personalised speech synthesis, which do not satisfy some practical application scenarios. Hence, an F-MelGAN model is adopted to improve the performance of Chinese speech synthesis. A post-processing network is used to refine the Mel-spectrum predicted by the decoder and alleviate the Mel-spectrum distortion phenomenon. A phoneme-level and sentence-level combined module is proposed to model the personalised style of speakers. A combination of an acoustic conditioning network, speaker encoder network GCNet and feedback-constrained training is proposed to solve the problem of poor personalised speech synthesis and achieve personalised speech customisation in Chinese. Experimental results show that the whole model can generate high-quality speech with high speaker similarity for both speakers that appear in the training process and speakers that never appear in the training process.</p>","PeriodicalId":34110,"journal":{"name":"IET Cybersystems and Robotics","volume":"5 3","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2023-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1049/csy2.12098","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IET Cybersystems and Robotics","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1049/csy2.12098","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

Speech interaction is an important means of robot interaction. With the rapid development of deep learning, end-to-end speech synthesis methods based on this technique have gradually become mainstream. Chinese deep learning-based speech synthesis techniques suffer from problems such as unstable synthesised speech, poor naturalness and poor personalised speech synthesis, which do not satisfy some practical application scenarios. Hence, an F-MelGAN model is adopted to improve the performance of Chinese speech synthesis. A post-processing network is used to refine the Mel-spectrum predicted by the decoder and alleviate the Mel-spectrum distortion phenomenon. A phoneme-level and sentence-level combined module is proposed to model the personalised style of speakers. A combination of an acoustic conditioning network, speaker encoder network GCNet and feedback-constrained training is proposed to solve the problem of poor personalised speech synthesis and achieve personalised speech customisation in Chinese. Experimental results show that the whole model can generate high-quality speech with high speaker similarity for both speakers that appear in the training process and speakers that never appear in the training process.

Abstract Image

用于机器人人机交互的中文个性化文本到语音合成
语音交互是机器人交互的重要手段。随着深度学习的快速发展,基于该技术的端到端语音合成方法逐渐成为主流。基于深度学习的汉语语音合成技术存在合成语音不稳定、自然度差、个性化语音合成差等问题,不能满足一些实际应用场景。因此,采用F-MelGAN模型来提高汉语语音合成的性能。后处理网络用于细化解码器预测的梅尔频谱,并缓解梅尔频谱失真现象。提出了一个音素级和句子级组合模块来对说话人的个性化风格进行建模。为了解决个性化语音合成较差的问题,实现中文个性化语音定制,提出了声学调节网络、说话人编码网络GCNet和反馈约束训练相结合的方法。实验结果表明,对于训练过程中出现的说话人和训练过程中从未出现的说话人,整个模型都可以生成具有高说话人相似度的高质量语音。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
IET Cybersystems and Robotics
IET Cybersystems and Robotics Computer Science-Information Systems
CiteScore
3.70
自引率
0.00%
发文量
31
审稿时长
34 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信