2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)最新文献_第9页

ASR Rescoring and Confidence Estimation with Electra 基于Electra的ASR评分及置信度估计

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-10-05 DOI: 10.1109/ASRU51503.2021.9688175

Hayato Futami, H. Inaguma, M. Mimura, S. Sakai, Tatsuya Kawahara

{"title":"ASR Rescoring and Confidence Estimation with Electra","authors":"Hayato Futami, H. Inaguma, M. Mimura, S. Sakai, Tatsuya Kawahara","doi":"10.1109/ASRU51503.2021.9688175","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688175","url":null,"abstract":"In automatic speech recognition (ASR) rescoring, the hypothesis with the fewest errors should be selected from the $n$-best list using a language model (LM). However, LMs are usually trained to maximize the likelihood of correct word sequences, not to detect ASR errors. We propose an ASR rescoring method for directly detecting errors with ELECTRA, which is originally a pre-training method for NLP tasks. ELECTRA is pre-trained to predict whether each word is replaced by BERT or not, which can simulate ASR error detection on large text corpora. To make this pre-training closer to ASR error detection, we further propose an extended version of ELECTRA called phone-attentive ELECTRA (P-ELECTRA). In the pre-training of P-ELECTRA, each word is replaced by a phone-to-word conversion model, which leverages phone information to generate acoustically similar words. Since our rescoring method is optimized for detecting errors, it can also be used for word-level confidence estimation. Experimental evaluations on the Librispeech and TED-LIUM2 corpora show that our rescoring method with ELECTRA is competitive with conventional rescoring methods with faster inference. ELECTRA also performs better in confidence estimation than BERT because it can learn to detect inappropriate words not only in fine-tuning but also in pre-training.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128507929","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 14

PL-EESR: Perceptual Loss Based End-to-End Robust Speaker Representation Extraction 基于感知损失的端到端鲁棒说话人表示提取

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-10-03 DOI: 10.1109/ASRU51503.2021.9688031

Yi Ma, Kong-Aik Lee, Ville Hautamäki, Haizhou Li

引用次数: 8

Tiny-CRNN: Streaming Wakeword Detection in a Low Footprint Setting Tiny-CRNN:低占用设置下的流唤醒词检测

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-29 DOI: 10.1109/ASRU51503.2021.9688299

Mohammad Omar Khursheed, Christin Jose, Rajath Kumar, Gengshen Fu, B. Kulis, S. Cheekatmalla

引用次数: 6

Comparison of Self-Supervised Speech Pre-Training Methods on Flemish Dutch 佛兰德语荷兰语自监督语音预训练方法比较

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-29 DOI: 10.1109/ASRU51503.2021.9688061

Jakob Poncelet, H. V. hamme

引用次数: 1

Multimodal Emotion Recognition with High-Level Speech and Text Features 基于高级语音和文本特征的多模态情感识别

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-29 DOI: 10.1109/ASRU51503.2021.9688036

M. R. Makiuchi, K. Uto, K. Shinoda

引用次数: 41

“How Robust R U?”: Evaluating Task-Oriented Dialogue Systems on Spoken Conversations “R U有多强大?”:评估以任务为导向的口语对话系统

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-28 DOI: 10.1109/ASRU51503.2021.9688274

Seokhwan Kim, Yang Liu, Di Jin, A. Papangelis, Karthik Gopalakrishnan, Behnam Hedayatnia, Dilek Z. Hakkani-Tür

引用次数: 29

Voxceleb Enrichment for Age and Gender Recognition Voxceleb年龄和性别识别浓缩

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-28 DOI: 10.1109/ASRU51503.2021.9688085

Khaled Hechmi, Trung Ngo Trong, Ville Hautamäki, T. Kinnunen

引用次数: 17

Word-Level Confidence Estimation for RNN Transducers RNN换能器的词水平置信度估计

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-28 DOI: 10.1109/ASRU51503.2021.9688247

Mingqiu Wang, H. Soltau, Laurent El Shafey, Izhak Shafran

{"title":"Word-Level Confidence Estimation for RNN Transducers","authors":"Mingqiu Wang, H. Soltau, Laurent El Shafey, Izhak Shafran","doi":"10.1109/ASRU51503.2021.9688247","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688247","url":null,"abstract":"Confidence estimate is an often requested feature in applications such as medical transcription where errors can impact patient care and the confidence estimate could be used to alert medical professionals to verify potential errors in recognition. In this paper, we present a lightweight neural confidence model tailored for Automatic Speech Recognition (ASR) system with Recurrent Neural Network Transducers (RNN-T). Compared to other existing approaches, our model utilizes: (a) the time information associated with recognized words, which reduces the computational complexity, and (b) a simple and elegant trick for mapping between sub-word and word sequences. The mapping addresses the non-unique tokenization and token deletion problems while amplifying differences between confusable words. Through extensive empirical evaluations on two different long-form test sets, we demonstrate that the model achieves a performance of 0.4 Normalized Cross Entropy (NCE) and 0.05 Expected Calibration Error (ECE). It is robust across different ASR configurations, including target types (graphemes vs. morphemes), traffic conditions (streaming vs. non-streaming), and encoder types. We further discuss the importance of evaluation metrics to reflect practical applications and highlight the need for further work in improving Area Under the Curve (AUC) for Negative Precision Rate (NPV) and True Negative Rate (TNR).","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"330 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122839915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Exploring Teacher-Student Learning Approach for Multi-Lingual Speech-to-Intent Classification 探索多语言语音-意图分类的师生学习方法

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-28 DOI: 10.1109/ASRU51503.2021.9688041

Bidisha Sharma, Maulik C. Madhavi, Xuehao Zhou, Haizhou Li

{"title":"Exploring Teacher-Student Learning Approach for Multi-Lingual Speech-to-Intent Classification","authors":"Bidisha Sharma, Maulik C. Madhavi, Xuehao Zhou, Haizhou Li","doi":"10.1109/ASRU51503.2021.9688041","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688041","url":null,"abstract":"End-to-end speech-to-intent classification has shown its advantage in harvesting information from both text and speech. In this paper, we study a technique to develop such an end-to-end system that supports multiple languages. To overcome the scarcity of multi-lingual speech corpus, we exploit knowledge from a pre-trained multi-lingual natural language processing model. Multi-lingual bidirectional encoder representations from transformers (mBERT) models are trained on multiple languages and hence expected to perform well in the multi-lingual scenario. In this work, we employ a teacher-student learning approach to sufficiently extract information from an mBERT model to train a multi-lingual speech model. In particular, we use synthesized speech generated from an English-Mandarin text corpus for analysis and training of a multi-lingual intent classification model. We also demonstrate that the teacher-student learning approach obtains an improved performance (91.02%) over the traditional end-to-end (89.40%) intent classification approach in a practical multi-lingual scenario.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"48 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120898694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates 基于非自回归隐中介的快速多解码器端到端语音翻译

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-09-27 DOI: 10.1109/ASRU51503.2021.9687894

H. Inaguma, Siddharth Dalmia, Brian Yan, Shinji Watanabe

{"title":"Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates","authors":"H. Inaguma, Siddharth Dalmia, Brian Yan, Shinji Watanabe","doi":"10.1109/ASRU51503.2021.9687894","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687894","url":null,"abstract":"The multi-decoder (MD) end-to-end speech translation model has demonstrated high translation quality by searching for better intermediate automatic speech recognition (ASR) decoder states as hidden intermediates (HI). It is a two-pass decoding model decomposing the overall task into ASR and machine translation sub-tasks. However, the decoding speed is not fast enough for real-world applications because it conducts beam search for both sub-tasks during inference. We propose Fast-MD, a fast MD model that generates HI by non-autoregressive (NAR) decoding based on connectionist temporal classification (CTC) outputs followed by an ASR decoder. We investigated two types of NAR HI: (1) parallel HI by using an autoregressive Transformer ASR decoder and (2) masked HI by using Mask-CTC, which combines CTC and the conditional masked language model. To reduce a mismatch in the ASR decoder between teacher-forcing during training and conditioning on CTC outputs during testing, we also propose sampling CTC outputs during training. Experimental evaluations on three corpora show that Fast-MD achieved about 2× and 4× faster decoding speed than that of the naïve MD model on GPU and CPU with comparable translation quality. Adopting the Conformer encoder and intermediate CTC loss further boosts its quality without sacrificing decoding speed.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128650601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7