2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)最新文献_第7页

Action Item Detection in Meetings Using Pretrained Transformers 在会议中使用预先训练的变压器进行行动项目检测

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688167

Kishan Sachdeva, Joshua Maynez, O. Siohan

{"title":"Action Item Detection in Meetings Using Pretrained Transformers","authors":"Kishan Sachdeva, Joshua Maynez, O. Siohan","doi":"10.1109/ASRU51503.2021.9688167","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688167","url":null,"abstract":"Detecting the action items that were agreed upon during a meeting has important practical applications. But, extremely sparse positive labels, noisy annotations, and small datasets limited the improvement in this task via hand-crafted features techniques. Given the breakthrough performance demonstrated by pretrained transformer-based models in a wide variety of NLP tasks, such as BERT [1] and ETC [2], we revisit this task using these modern techniques. We empirically show how these modelling techniques advance the state-of-the-art on action item detection for the ICSI simulated meeting corpus [3] by 75% and establish a baseline for the action item detection from the AMI [4] meeting corpus. We show that these models are competitive on the related ICSI MRDA classification problem. In order to push the performance even further, we re-evaluate the task definition for action item detection, drawing upon the similarities with the span boundary detection realm. We propose the use of Generalized Hamming Distance ghd as an alternative evaluation. We hope to motivate further interest into the action item detection task by the community.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123383971","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Improving ASR Error Correction Using N-Best Hypotheses 基于n -最优假设的ASR纠错改进

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688210

Linchen Zhu, Wenjie Liu, Linquan Liu, Ed Lin

{"title":"Improving ASR Error Correction Using N-Best Hypotheses","authors":"Linchen Zhu, Wenjie Liu, Linquan Liu, Ed Lin","doi":"10.1109/ASRU51503.2021.9688210","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688210","url":null,"abstract":"In the field of Automatic Speech Recognition (ASR), Grammatical Error Correction (GEC) can be used to correct errors in recognition results of ASR systems and whereby it further reduces the word error rate (WER). Most conventional GEC approaches make correction merely based on a single recognition hypothesis from the ASR system, i.e., the best path in the decoding lattice. However, with such limited information, it's common that GEC fails to detect errors in a sentence or even makes wrong corrections. In this paper we propose two novel GEC approaches utilizing N-best hypotheses instead, that's, N-best plain approach and N-best alignment approach. In this way, our novel approaches can access more information than conventional approaches with 1-best-only hypothesis, and thus more accurate corrections can be made. Experiments on an industrial-scale internal dataset and the open-source dataset AISHELL-1 demonstrate the effectiveness of N-best approaches: 1) both N-best plain and N-best alignment approaches achieve remarkable WER reduction. N-best alignment approach achieves the best performance on both datasets, 12.54% WERR on an internal dataset and 27.71% WERR on AISHELL-1. 2) They also outperform the baseline 1-best GEC approach by a large margin.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129468202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

Ensemble of Domain Adversarial Neural Networks for Speech Emotion Recognition 面向语音情感识别的领域对抗神经网络集成

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687905

Shi-wook Lee

{"title":"Ensemble of Domain Adversarial Neural Networks for Speech Emotion Recognition","authors":"Shi-wook Lee","doi":"10.1109/ASRU51503.2021.9687905","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687905","url":null,"abstract":"Domain-invariant class-discriminative representation across multiple emotional speech corpora is essential for multi-corpus speech emotion recognition. When training a model using multiple corpora, the generalization ability must be considered by hindering the memorization of domain-specific information, and recognition should be performed on features that cannot discriminate among domains. First, the performance degradation of multi-corpus speech emotion recognition systems was investigated, and a novel frame-work adopting an ensemble of several domain adversarial neural networks was proposed. Experiments were conducted on single-corpus and multi-corpus systems for two corpora, namely an English corpus (IEMOCAP) and a Japanese corpus (JTES). The findings of the experiments revealed that although individual domain adversarial neural networks exhibit information loss and inferior performance, an excellent comeback accuracy can be achieved by an ensemble of these individual domain adversarial neural networks. We confirmed that the proposed framework effectively learned both domain-invariant and class-discriminative representations.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"224 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121621242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Improving Speech Recognition on Noisy Speech via Speech Enhancement with Multi-Discriminators CycleGAN 基于多鉴别器周期gan的语音增强改进对噪声语音的识别

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-12 DOI: 10.1109/ASRU51503.2021.9688310

Chia-Yu Li, Ngoc Thang Vu

引用次数: 4

Hierarchical Knowledge Distillation for Dialogue Sequence Labeling 对话序列标注的层次知识精馏

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-11-22 DOI: 10.1109/ASRU51503.2021.9687959

Shota Orihashi, Yoshihiro Yamazaki, Naoki Makishima, Mana Ihori, Akihiko Takashima, Tomohiro Tanaka, Ryo Masumura

{"title":"Hierarchical Knowledge Distillation for Dialogue Sequence Labeling","authors":"Shota Orihashi, Yoshihiro Yamazaki, Naoki Makishima, Mana Ihori, Akihiko Takashima, Tomohiro Tanaka, Ryo Masumura","doi":"10.1109/ASRU51503.2021.9687959","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687959","url":null,"abstract":"This paper presents a novel knowledge distillation method for dialogue sequence labeling. Dialogue sequence labeling is a supervised learning task that estimates labels for each utterance in the target dialogue document, and is useful for many applications such as dialogue act estimation. Accurate labeling is often realized by a hierarchically-structured large model consisting of utterance-level and dialogue-level networks that capture the contexts within an utterance and between utterances, respectively. However, due to its large model size, such a model cannot be deployed on resource-constrained devices. To overcome this difficulty, we focus on knowledge distillation which trains a small model by distilling the knowledge of a large and high performance teacher model. Our key idea is to distill the knowledge while keeping the complex contexts captured by the teacher model. To this end, the proposed method, hierarchical knowledge distillation, trains the small model by distilling not only the probability distribution of the label classification, but also the knowledge of utterance-level and dialogue-level contexts trained in the teacher model by training the model to mimic the teacher model's output in each level. Experiments on dialogue act estimation and call scene segmentation demonstrate the effectiveness of the proposed method.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"119 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125204666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Comparison of Streaming Models and Data Augmentation Methods for Robust Speech Recognition 鲁棒语音识别的流模型与数据增强方法比较

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-11-19 DOI: 10.1109/ASRU51503.2021.9688002

Jiyeon Kim, Mehul Kumar, Dhananjaya N. Gowda, Abhinav Garg, Chanwoo Kim

{"title":"A Comparison of Streaming Models and Data Augmentation Methods for Robust Speech Recognition","authors":"Jiyeon Kim, Mehul Kumar, Dhananjaya N. Gowda, Abhinav Garg, Chanwoo Kim","doi":"10.1109/ASRU51503.2021.9688002","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688002","url":null,"abstract":"In this paper, we present a comparative study on the robustness of two different online streaming speech recognition models: Monotonic Chunkwise Attention (MoChA) and Recurrent Neural Network-Transducer (RNN-T). We explore three recently proposed data augmentation techniques, namely, multi-conditioned training using an acoustic simulator, Vocal Tract Length Perturbation (VTLP) for speaker variability, and SpecAugment. Experimental results show that unidirectional models are in general more sensitive to noisy examples in the training set. It is observed that the final performance of the model depends on the proportion of training examples processed by data augmentation techniques. MoChA models generally perform better than RNN-T models. However, we observe that training of MoChA models seems to be more sensitive to various factors such as the characteristics of training sets and the incorporation of additional augmentations techniques. On the other hand, RNN-T models perform better than MoChA models in terms of latency, inference time, and the stability of training. Additionally, RNN-T models are generally more robust against noise and reverberation. All these advantages make RNN-T models a better choice for streaming on-device speech recognition compared to MoChA models.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133883382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Semi-Supervised Transfer Learning for Language Expansion of End-to-End Speech Recognition Models to Low-Resource Languages 基于半监督迁移学习的端到端语音识别模型的语言扩展

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-11-19 DOI: 10.1109/ASRU51503.2021.9688019

Jiyeon Kim, Mehul Kumar, Dhananjaya N. Gowda, Abhinav Garg, Chanwoo Kim

{"title":"Semi-Supervised Transfer Learning for Language Expansion of End-to-End Speech Recognition Models to Low-Resource Languages","authors":"Jiyeon Kim, Mehul Kumar, Dhananjaya N. Gowda, Abhinav Garg, Chanwoo Kim","doi":"10.1109/ASRU51503.2021.9688019","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688019","url":null,"abstract":"In this paper, we propose a three-stage training methodology to improve the speech recognition accuracy of low-resource languages. We explore and propose an effective combination of techniques such as transfer learning, encoder freezing, data augmentation using Text-To-Speech (TTS), and Semi-Supervised Learning (SSL). To improve the accuracy of a low-resource Italian ASR, we leverage a well-trained English model, unlabeled text corpus, and unlabeled audio corpus using transfer learning, TTS augmentation, and SSL respectively. In the first stage, we use transfer learning from a well-trained English model. This primarily helps in learning the acoustic information from a resource-rich language. This stage achieves around 24% relative Word Error Rate (WER) reduction over the baseline. In stage two, We utilize unlabeled text data via TTS data-augmentation to incorporate language information into the model. We also explore freezing the acoustic encoder at this stage. TTS data augmentation helps us further reduce the WER by ∼ 21 % relatively. Finally, In stage three we reduce the WER by another 4% relative by using SSL from unlabeled audio data. Overall, our two-pass speech recognition system with a Monotonic Chunkwise Attention (MoChA) in the first pass and a full-attention in the second pass achieves a WER reduction of ∼ 42% relative to the baseline.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121148460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

A Conformer-Based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation 一种基于调形器的联合声回波消除、语音增强和语音分离ASR前端

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-11-18 DOI: 10.1109/ASRU51503.2021.9687942

Tom O'Malley, A. Narayanan, Quan Wang, A. Park, James Walker, N. Howard

{"title":"A Conformer-Based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation","authors":"Tom O'Malley, A. Narayanan, Quan Wang, A. Park, James Walker, N. Howard","doi":"10.1109/ASRU51503.2021.9687942","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687942","url":null,"abstract":"We present a frontend for improving robustness of automatic speech recognition (ASR), that jointly implements three modules within a single model: acoustic echo cancellation, speech enhancement, and speech separation. This is achieved by using a contextual enhancement neural network that can optionally make use of different types of side inputs: (1) a reference signal of the playback audio, which is necessary for echo cancellation; (2) a noise context, which is useful for speech enhancement; and (3) an embedding vector representing the voice characteristic of the target speaker of interest, which is not only critical in speech separation, but also helpful for echo cancellation and speech enhancement. We present detailed evaluations to show that the joint model performs almost as well as the task-specific models, and significantly reduces word error rate in noisy conditions even when using a large-scale state-of-the-art ASR model. Compared to the noisy baseline, the joint model reduces the word error rate in low signal-to-noise ratio conditions by at least 71% on our echo cancellation dataset, 10% on our noisy dataset, and 26% on our multi-speaker dataset. Compared to task-specific models, the joint model performs within 10% on our echo cancellation dataset, 2% on the noisy dataset, and 3% on the multi-speaker dataset.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129857066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 21

Attention-Based Multi-Hypothesis Fusion for Speech Summarization 基于注意力的语音摘要多假设融合

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-11-16 DOI: 10.1109/ASRU51503.2021.9687977

Takatomo Kano, A. Ogawa, Marc Delcroix, Shinji Watanabe

{"title":"Attention-Based Multi-Hypothesis Fusion for Speech Summarization","authors":"Takatomo Kano, A. Ogawa, Marc Delcroix, Shinji Watanabe","doi":"10.1109/ASRU51503.2021.9687977","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687977","url":null,"abstract":"Speech summarization, which generates a text summary from speech, can be achieved by combining automatic speech recognition (ASR) and text summarization (TS). With this cascade approach, we can exploit state-of-the-art models and large training datasets for both subtasks, i.e., Transformer for ASR and Bidirectional Encoder Representations from Transformers (BERT) for TS. However, ASR errors directly affect the quality of the output summary in the cascade approach. We propose a cascade speech summarization model that is robust to ASR errors and that exploits multiple hypotheses generated by ASR to attenuate the effect of ASR errors on the summary. We investigate several schemes to combine ASR hypotheses. First, we propose using the sum of sub-word embedding vectors weighted by their posterior values provided by an ASR system as an input to a BERT-based TS system. Then, we introduce a more general scheme that uses an attention-based fusion module added to a pre-trained BERT module to align and combine several ASR hypotheses. Finally, we perform speech summarization experiments on the How2 dataset and a newly assembled TED-based dataset that we will release with this paper11https://github.com/nttcslab-sp-admin/TEDSummary. These experiments show that retraining the BERT-based TS system with these schemes can improve summarization performance and that the attention-based fusion module is particularly effective.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132607921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

AC-VC: Non-Parallel Low Latency Phonetic Posteriorgrams Based Voice Conversion 基于非并行低延迟语音后图的语音转换

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-11-12 DOI: 10.1109/ASRU51503.2021.9688277

Damien Ronssin, M. Cernak

引用次数: 9