11th ISCA Speech Synthesis Workshop (SSW 11)最新文献_第2页

Factors Affecting the Evaluation of Synthetic Speech in Context 语境中影响合成语音评价的因素

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-26

Johannah O'Mahony, Pilar Oplustil-Gallegos, Catherine Lai, Simon King

{"title":"Factors Affecting the Evaluation of Synthetic Speech in Context","authors":"Johannah O'Mahony, Pilar Oplustil-Gallegos, Catherine Lai, Simon King","doi":"10.21437/ssw.2021-26","DOIUrl":"https://doi.org/10.21437/ssw.2021-26","url":null,"abstract":"Text-to-Speech synthesis is approaching the limit of naturalness that is possible from an isolated sentence. The focus of research is shifting to modelling contextual information, typically with the goal of producing better prosodic realisations by accounting for longer-range text dependencies from preceding sentences. But current evaluation methods were developed for single sentences and it is not yet clear how the evaluation of longer texts should be approached. Previous work suggests that evaluation of utterances in context can lead to an increase in Mean Opinion Score ratings, even when the synthesis technique is not context-aware. We investigated several factors that might explain this increase. Three experiments manipulated: the wording of instructions that participants received; the textual characteristics of context-stimulus pairs; and the prosodic realisation of the synthetic speech. We found that the wording of instructions has an impact on listeners’ ratings of stimuli presented in context. The between-sentence context dependency of stimulus text has no impact on ratings. Listeners are, however, sensitive to prosodic differences, both in context and in isolation.","PeriodicalId":106906,"journal":{"name":"11th ISCA Speech Synthesis Workshop (SSW 11)","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115808888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Lipsyncing efforts for transcreating lecture videos in Indian languages 对口型的努力，在印度语言的翻译讲座视频

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-38

M. ManoRanjithKumar, Jom Kuriakose, S. KarthikPandiaD., H. Murthy

{"title":"Lipsyncing efforts for transcreating lecture videos in Indian languages","authors":"M. ManoRanjithKumar, Jom Kuriakose, S. KarthikPandiaD., H. Murthy","doi":"10.21437/ssw.2021-38","DOIUrl":"https://doi.org/10.21437/ssw.2021-38","url":null,"abstract":"This paper proposes a novel lip-syncing module for the transcreation of lecture videos from English to Indian languages. The audio from the lecture is transcribed using automatic speech recognition. The text is translated and manually curated before and after translation to avoid mistakes. The curated text is synthesized using the Indian language end-to-end-based text-to-speech synthesis systems. The synthesized audio and video are out-of-sync. This paper attempts to automate this process of producing video lectures lip-synced into Indian languages using different techniques. Lip-syncing an educational video with the Indian language audio is challenging owing to (a) the duration of Indian language audio being considerably longer or shorter than that of the original audio, (b) the extempore speech causes the audio in the source videos to have long silences. Any modiﬁcation to the speed of audio can be unpleasant to listeners. The proposed system non-uniformly re-samples the video to ensure better lip-syncing. The novelty of this paper is in the use of HMM-GMM alignments in tandem with syllable segmentation using group delay, as visemes are closer to syllables. The proposed lip-syncing techniques are evaluated using subjective evaluation methods. Results indicate that accurate alignment at the syllable level is crucial for lip-syncing.","PeriodicalId":106906,"journal":{"name":"11th ISCA Speech Synthesis Workshop (SSW 11)","volume":"44 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125677836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Acquiring conversational speaking style from multi-speaker spontaneous dialog corpus for prosody-controllable sequence-to-sequence speech synthesis 从多说话人自发对话语料库中获取会话式说话风格，用于韵律可控序对序语音合成

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-12

Slava Shechtman, Avrech Ben-David

{"title":"Acquiring conversational speaking style from multi-speaker spontaneous dialog corpus for prosody-controllable sequence-to-sequence speech synthesis","authors":"Slava Shechtman, Avrech Ben-David","doi":"10.21437/ssw.2021-12","DOIUrl":"https://doi.org/10.21437/ssw.2021-12","url":null,"abstract":"Sequence-to-Sequence Text-to-Speech (S2S TTS) architectures that directly generate low level acoustic features from phonetic sequence are known to produce natural and expressive speech, when provided with moderate-to-large amounts of high quality training data. When exposed to a sequence of coarse speaker-agnostic prosodic descriptors , such systems become prosody-controllable and can learn and transfer desired prosodic patterns (e.g. word-emphasis or speaking style) from one seen speaker to another (in multi-speaker settings). But what if a high quality speech corpus for a desired speaking style is not available? In this work we explore the feasibility of teaching a neutral pre-trained prosody-controllable S2S TTS voice to speak with a conversational speaking style, as learnt from a low-quality multi-speaker spontaneous dialog corpus (originally intended for Automatic Speech Recognition). We have found that it is absolutely necessary to incorporate word semantics for that task. We ﬁne-tune BERT network to predict the prosodic descriptors from the input text, based on that corpus, and apply them to the prosody-controllable S2S TTS at inference time. The subjective listening tests revealed that the learnt conversational style rated higher than baseline for 68% of the stimuli under test. The overall quality and naturalness rated higher than baseline in 64% of the stimuli under test. The improvement came mostly as a result of improving common conversational speech patterns, such as ﬁller words and phrases. However, the overall MOS did not signiﬁcantly improve due to less convincing realization of the rising intonation on declarative statements ( uptalk ).","PeriodicalId":106906,"journal":{"name":"11th ISCA Speech Synthesis Workshop (SSW 11)","volume":"99 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124921988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Liaison and Pronunciation Learning in End-to-End Text-to-Speech in French 法语端到端文本到语音的连读和发音学习

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-34

Jason Taylor, Sébastien Le Maguer, Korin Richmond

{"title":"Liaison and Pronunciation Learning in End-to-End Text-to-Speech in French","authors":"Jason Taylor, Sébastien Le Maguer, Korin Richmond","doi":"10.21437/ssw.2021-34","DOIUrl":"https://doi.org/10.21437/ssw.2021-34","url":null,"abstract":"Sequence-to-sequence (S2S) TTS models like Tacotron have grapheme-only inputs when trained fully end-to-end. Grapheme inputs map to phone sounds depending on context, which traditionally is handled by extensive preprocessing in the TTS front-end. However, French orthography does not provide a clear one-to-one mapping between graphemes and sounds, and in English, which similarly has rather non-phonetic orthography, pronunciations are a signiﬁcant cause of error in S2S-TTS with grapheme-inputs. In this paper, we test implicit pronunciation knowledge where graphemes do not map directly to phones. Implicit pronunciation knowledge learnt in S2S-TTS is similar to a standalone grapheme-to-phoneme (G2P) model, which makes explicit phone predictions at the sequential level. We ﬁnd grapheme-input S2S-TTS makes implicit pronunciation errors similar to explicit G2P models - notably for foreign names. In a traditional front-end pipeline, there are also post-lexical rules which modify G2P output at the sequential level. In French, post-lexical rules require a deep knowledge of linguistic structure in a process called Liaison . Without explicit rules, we ﬁnd S2S-TTS with grapheme-inputs over-inserts Liaison sounds, leading to a signiﬁcant preference for a phone-based equivalent. By testing with linguistically-motivated stimuli, we observe differences that would otherwise go undetected.","PeriodicalId":106906,"journal":{"name":"11th ISCA Speech Synthesis Workshop (SSW 11)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130145877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Are we truly modeling expressiveness? A study on expressive TTS in Brazilian Portuguese for real-life application styles 我们真的在塑造表达能力吗?巴西葡萄牙语表达性TTS的现实应用风格研究

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-15

Lucas H. Ueda, P. D. P. Costa, F. Simões, M. U. Neto

引用次数: 0

Perception of smiling voice in spontaneous speech synthesis 自发语音合成中微笑声音的感知

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-19

Ambika Kirkland, M. Wlodarczak, Joakim Gustafson, Éva Székely

{"title":"Perception of smiling voice in spontaneous speech synthesis","authors":"Ambika Kirkland, M. Wlodarczak, Joakim Gustafson, Éva Székely","doi":"10.21437/ssw.2021-19","DOIUrl":"https://doi.org/10.21437/ssw.2021-19","url":null,"abstract":"Smiling during speech production has been shown to result in perceptible acoustic differences compared to non-smiling speech. However, there is a scarcity of research on the perception of “smiling voice” in synthesized spontaneous speech. In this study, we used a sequence-to-sequence neural text-to-speech system built on conversational data to produce utterances with the characteristics of spontaneous speech. Segments of speech following laughter, and the same utterances not preceded by laughter, were compared in a perceptual experiment after removing laughter and/or breaths from the beginning of the utterance to determine whether participants perceive the utterances preceded by laughter as sounding as if they were produced while smiling. The results showed that participants identiﬁed the post-laughter speech as smiling at a rate signif-icantly greater than chance. Furthermore, the effect of content (positive/neutral/negative) was investigated. These results show that laughter, a spontaneous, non-elicited phenomenon in our model’s training data, can be used to synthesize expressive speech with the perceptual characteristics of smiling.","PeriodicalId":106906,"journal":{"name":"11th ISCA Speech Synthesis Workshop (SSW 11)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133807025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Cross-lingual Transfer of Phonological Features for Low-resource Speech Synthesis 低资源语音合成中语音特征的跨语迁移

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-28

Dan Wells, Korin Richmond

引用次数: 7

Vocal tract area function extraction using ultrasound for articulatory speech synthesis 超声声道区域功能提取用于发音语音合成

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-16

Debasis Mohapatra, Pramit Saha, Yadong Liu, B. Gick, S. Fels

{"title":"Vocal tract area function extraction using ultrasound for articulatory speech synthesis","authors":"Debasis Mohapatra, Pramit Saha, Yadong Liu, B. Gick, S. Fels","doi":"10.21437/ssw.2021-16","DOIUrl":"https://doi.org/10.21437/ssw.2021-16","url":null,"abstract":"This paper studies the feasibility of an articulatory speech synthesizer by extracting the mid-sagittal tongue and palate contours using the ultrasound (US) imaging modality. The extracted contours are then used to compute the vocal tract cross-sectional areas (i.e., area function) during phonation, which then drives an articulary speech synthesizer. Using this approach, we synthesized four phonetic vowel sounds (/a/, /i/, /e/ and /o/). The derived vocal tract (VT) transfer functions are shown to match over multiple utterances for a single vowel, thereby conﬁrming reliable and accurate area function deriva-tion using the US. The acoustic formants of simulated vowels using the proposed method show a modest deviation from the speaker’s recorded speech signal since the current articulatory model does not include the mouth radiation mechanism. Furthermore, the higher formants’ positions (F5-F8) are approximately equivalent to the high-quality standard MRI-based acoustic results and have an average error of 3 . 90% , 4 . 14% , 1 . 26% and 2 . 99% for vowel sounds /a/, /i/, /e/ and /o/, respectively. Our approach provides a step towards developing a US-based speech synthesizer for precise extraction of the upper VT geometry and enabling speakers to drive an articulatory model directly by their tongue movements without the necessity of vo-calization.","PeriodicalId":106906,"journal":{"name":"11th ISCA Speech Synthesis Workshop (SSW 11)","volume":"36 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123287623","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Analysing Temporal Sensitivity of VQ-VAE Sub-Phone Codebooks VQ-VAE子电话码本时间灵敏度分析

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-40

Jason Fong, Jennifer Williams, Simon King

引用次数: 1

Improving Polyglot Speech Synthesis through Multi-task and Adversarial Learning 通过多任务和对抗学习改进多语语音合成

11th ISCA Speech Synthesis Workshop (SSW 11) Pub Date : 2021-08-26 DOI: 10.21437/ssw.2021-30

Jason Fong, Jilong Wu, Prabhav Agrawal, Andrew Gibiansky, T. Koehler, Qing He

引用次数: 2