2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)最新文献_第5页

Variational Sequential Modeling, Learning and Understanding 变分序列建模，学习和理解

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687925

Jen-Tzung Chien, Chih-Jung Tsai

{"title":"Variational Sequential Modeling, Learning and Understanding","authors":"Jen-Tzung Chien, Chih-Jung Tsai","doi":"10.1109/ASRU51503.2021.9687925","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687925","url":null,"abstract":"Normalizing flow comprises of a series of invertible transformations. With careful design in transformations, it can generate images or speeches with fast sampling speed. Inference can also be efficient in maximum likelihood manner. In addition to generating scenes or human faces, it can be used to transform probability distributions. On the other hand, learning latent structures of sentences in a global manner is always challenging. Variational autoencoder (VAE) is haunted by the issue of posterior collapse, where the latent space is poorly learned. To improve inference and generation of VAE in learning sequence data, we propose the amortized flow posterior variational recurrent autoencoder (AFP-VRAE). Variational recurrent autoencoder (VRAE) has RNN based encoder and decoder and learns global representations of sentences. To learn latent space that well preserves the semantic information of data, we use the normalizing flow to generate flexible variational distributions. Furthermore, we adopt the amortized regularization to encode similar embeddings to neighboring latent representations, and we use the skip connections to reinforce the representations to predict every output directly. The benefits can be shown in the experiments as we evaluate the models for language modeling, sentiment analysis and document summarization. AFP-VRAE reports good results on variational modeling for sequence data.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128339263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Data Augmentation for ASR Using TTS Via a Discrete Representation 基于离散表示的TTS的ASR数据增强

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688218

Sei Ueno, M. Mimura, S. Sakai, Tatsuya Kawahara

{"title":"Data Augmentation for ASR Using TTS Via a Discrete Representation","authors":"Sei Ueno, M. Mimura, S. Sakai, Tatsuya Kawahara","doi":"10.1109/ASRU51503.2021.9688218","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688218","url":null,"abstract":"While end-to-end automatic speech recognition (ASR) has achieved high performance, it requires a huge amount of paired speech and transcription data for training. Recently, data augmentation methods have actively been investigated. One method is to use a text-to-speech (TTS) system to gen-erate speech data from text-only data and use the generated speech for data augmentation, but it has been found that the synthesized log Mel-scale filterbank (lmfb) features could have a serious mismatch with the real speech features. In this study, we propose a data augmentation method via a discrete speech representation. The TTS model predicts discrete ID sequences instead of lmfb features, and the ASR also uses the ID sequences as training data. We expect that the use of a discrete representation based on vq-wav2vec not only makes TTS training easier but also mitigates the mismatch with real data. Experimental evaluations show that the pro-posed method outperforms the data augmentation method using the conventional TTS. We found that it reduces speaker dependency, and the generated features are distributed more closely to the real ones.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116858229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11

Estimating the Generation Timing of Responsive Utterances by Active Listeners of Spoken Narratives 口述叙事的主动听者反应性话语产生时间的估计

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688132

Koichiro Ito, M. Murata, T. Ohno, S. Matsubara

引用次数: 0

Dialogue Strategy Adaptation to New Action Sets Using Multi-Dimensional Modelling 对话策略适应新行动集使用多维建模

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688038

Simon Keizer, N. Braunschweiler, Svetlana Stoyanchev, R. Doddipatla

{"title":"Dialogue Strategy Adaptation to New Action Sets Using Multi-Dimensional Modelling","authors":"Simon Keizer, N. Braunschweiler, Svetlana Stoyanchev, R. Doddipatla","doi":"10.1109/ASRU51503.2021.9688038","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688038","url":null,"abstract":"A major bottleneck for building statistical spoken dialogue systems for new domains and applications is the need for large amounts of training data. To address this problem, we adopt the multi-dimensional approach to dialogue management and evaluate its potential for transfer learning. Specifically, we exploit pre-trained task-independent policies to speed up training for an extended task-specific action set, in which the single summary action for requesting a slot is replaced by multiple slot-specific request actions. Policy optimisation and evaluation experiments using an agenda-based user simulator show that with limited training data, much better performance levels can be achieved when using the proposed multi-dimensional adaptation method. We confirm this improvement in a crowd-sourced human user evaluation of our spoken dialogue system, comparing partially trained policies. The multi-dimensional system (with adaptation on limited training data in the target scenario) outperforms the one-dimensional baseline (without adaptation on the same amount of training data) by 7% perceived success rate.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117216411","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training TGAVC:通过文本引导和对抗性训练改进自动编码器语音转换

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688088

Huaizhen Tang, Xulong Zhang, Jianzong Wang, Ning Cheng, Zhen Zeng, Edward Xiao, Jing Xiao

{"title":"TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training","authors":"Huaizhen Tang, Xulong Zhang, Jianzong Wang, Ning Cheng, Zhen Zeng, Edward Xiao, Jing Xiao","doi":"10.1109/ASRU51503.2021.9688088","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688088","url":null,"abstract":"Non-parallel many-to-many voice conversion remains an interesting but challenging speech processing task. Recently, AutoVC, a conditional autoencoder based method, achieved excellent conversion results by disentangling the speaker identity and the speech content using information-constraining bottlenecks. However, due to the pure autoencoder training method, it is difficult to evaluate the separation effect of content and speaker identity. In this paper, a novel voice conversion framework, named Text Guided AutoVC(TGAVC), is proposed to more effectively separate content and timbre from speech, where an expected content embedding produced based on the text transcriptions is designed to guide the extraction of voice content. In addition, the adversarial training is applied to eliminate the speaker identity information in the estimated content embedding extracted from speech. Under the guidance of the expected content embedding and the adversarial training, the content encoder is trained to extract speaker-independent content embedding from speech. Experiments on AIShell-3 dataset show that the proposed model outperforms AutoVC in terms of naturalness and similarity of converted speech.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"279 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115596371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

Vibrato Learning in Multi-Singer Singing Voice Synthesis 多歌手演唱声音合成中的颤音学习

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688029

Ruolan Liu, Xue Wen, Chunhui Lu, Liming Song, June Sig Sung

{"title":"Vibrato Learning in Multi-Singer Singing Voice Synthesis","authors":"Ruolan Liu, Xue Wen, Chunhui Lu, Liming Song, June Sig Sung","doi":"10.1109/ASRU51503.2021.9688029","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688029","url":null,"abstract":"Decent vibratos are a trait of good vocal training, often associated with perceived level of singing skill. In this paper we present a system for multi-singer singing voice synthesis, which is capable of producing high quality singing with convincing, controllable vibratos, and can also synthesize natural singing voices for target speakers with only speech data. This is enabled by using a unified speech-and-singing acoustic model that not only bridges the modality gap but also helps make best use of both types of data. The acoustic model exposes the full F0 contour therefore allowing explicitly modelling of F0 characteristics specific to singing voice. We observe that short-time Fourier transform of the F0 contour sparsely encode vibrato characteristics, and derive a learning objective therefrom for improved vibrato production. Control of the synthesized vibrato extent is possible by wiring in a supervised “extent” neuron and expose it to outer system. Experimental results confirm the effectiveness of proposed objective in producing good vibratos and improving overall perceived singing voice quality.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"105 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115793784","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Two-Pass End-to-End ASR Model Compression 两路端到端ASR模型压缩

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688121

Nauman Dawalatabad, Tushar Vatsal, Ashutosh Gupta, Sungsoo Kim, Shatrughan Singh, Dhananjaya N. Gowda, Chanwoo Kim

{"title":"Two-Pass End-to-End ASR Model Compression","authors":"Nauman Dawalatabad, Tushar Vatsal, Ashutosh Gupta, Sungsoo Kim, Shatrughan Singh, Dhananjaya N. Gowda, Chanwoo Kim","doi":"10.1109/ASRU51503.2021.9688121","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688121","url":null,"abstract":"Speech recognition on smart devices is challenging owing to the small memory footprint. Hence small size ASR models are desirable. With the use of popular transducer-based models, it has become possible to practically deploy streaming speech recognition models on small devices [1]. Recently, the two-pass model [2] combining RNN-T and LAS modules has shown exceptional performance for streaming on-device speech recognition. In this work, we propose a simple and effective approach to reduce the size of the two-pass model for memory-constrained devices. We employ a popular knowledge distillation approach in three stages using the Teacher-Student training technique. In the first stage, we use a trained RNN-T model as a teacher model and perform knowledge distillation to train the student RNN-T model. The second stage uses the shared encoder and trains a LAS rescorer for student model using the trained RNN-T+LAS teacher model. Finally, we perform deep-finetuning for the student model with a shared RNN-T encoder, RNN-T decoder, and LAS rescorer. Our experimental results on standard LibriSpeech dataset show that our system can achieve a high compression rate of 55% without significant degradation in the WER compared to the two-pass teacher model.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132117421","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

A Study on Cross-Corpus Speech Emotion Recognition and Data Augmentation 跨语料库语音情感识别及数据增强研究

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687987

N. Braunschweiler, R. Doddipatla, Simon Keizer, Svetlana Stoyanchev

{"title":"A Study on Cross-Corpus Speech Emotion Recognition and Data Augmentation","authors":"N. Braunschweiler, R. Doddipatla, Simon Keizer, Svetlana Stoyanchev","doi":"10.1109/ASRU51503.2021.9687987","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687987","url":null,"abstract":"Models that can handle a wide range of speakers and acoustic conditions are essential in speech emotion recognition (SER). Often, these models tend to show mixed results when presented with speakers or acoustic conditions that were not visible during training. This paper investigates the impact of cross-corpus data complementation and data augmentation on the performance of SER models in matched (test-set from same corpus) and mismatched (test-set from different corpus) conditions. Investigations using six emotional speech corpora that include single and multiple speakers as well as variations in emotion style (acted, elicited, natural) and recording conditions are presented. Observations show that, as expected, models trained on single corpora perform best in matched conditions while performance decreases between 10-40% in mismatched conditions, depending on corpus specific features. Models trained on mixed corpora can be more stable in mismatched contexts, and the performance reductions range from 1 to 8% when compared with single corpus models in matched conditions. Data augmentation yields additional gains up to 4% and seem to benefit mismatched conditions more than matched ones.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131079743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Target Language Extraction at Multilingual Cocktail Parties 多语种鸡尾酒会的目标语言提取

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688052

Marvin Borsdorf, Haizhou Li, T. Schultz

引用次数: 4

Improving Reverberant Speech Separation with Synthetic Room Impulse Responses 利用合成房间脉冲响应改进混响语音分离

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688203

R. Aralikatti, Anton Ratnarajah, Zhenyu Tang, Dinesh Manocha

引用次数: 5