2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)最新文献_第3页

Automatic Generation of Diagnostic Content Feedback in Spoken Language Learning and Assessment 口语学习与评估中诊断性内容反馈的自动生成

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688207

Xinhao Wang, Chris Hamill

{"title":"Automatic Generation of Diagnostic Content Feedback in Spoken Language Learning and Assessment","authors":"Xinhao Wang, Chris Hamill","doi":"10.1109/ASRU51503.2021.9688207","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688207","url":null,"abstract":"This study aims to explore an effective way to automati-cally generate actionable diagnostic feedback targeting the content development skill within the context of spoken language learning and assessment. There is one type of test question widely used in speaking assessment, which requires test takers to first listen to and/or read stimulus material and then create a spontaneous response to a question related to the stimulus. In a high-proficiency response, critical content from the stimulus - referred to as “key points” - should be properly covered. We propose Transformer-based models to automatically detect the location of key point spans within a response, or if no key points are covered, to detect their absence. Additionally, we introduce a multi-task learning approach for assigning a “quality score” to each key point span, measuring how well the key point is rendered within the response. Experimental results demonstrate that the pro-posed models can surpass human expert performance based on human transcriptions. In contrast, based on automatic speech recognition hypotheses, models underperform human expert performance with an F1-score at 66.6% (vs. human agreement of 68.8%) on span detection and a Pearson corre-lation coefficient at 0.579 (vs. human agreement of 0.610) on quality score prediction.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130474829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning How Long to Wait: Adaptively-Constrained Monotonic Multihead Attention for Streaming ASR 学习等待时间:流ASR的自适应约束单调多头注意

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688138

Jae-gyun Song, Hajin Shim, Eunho Yang

引用次数: 0

Learning to Translate Low-Resourced Swiss German Dialectal Speech into Standard German Text 学习将资源贫乏的瑞士德语方言语音翻译成标准德语文本

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688249

Abbas Khosravani, Philip N. Garner, Alexandros Lazaridis

{"title":"Learning to Translate Low-Resourced Swiss German Dialectal Speech into Standard German Text","authors":"Abbas Khosravani, Philip N. Garner, Alexandros Lazaridis","doi":"10.1109/ASRU51503.2021.9688249","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688249","url":null,"abstract":"For a low-resourced language like Swiss German with no standard orthography and a significant variation in its written form, spoken language resources are more likely to come with translations than transcriptions. Moreover, the desired output of an automatic transcription system for Swiss German multi-dialectal speech is Standard German. This, in turn, is due to many applications that include our TV Box voice assistant and broadcast media. It follows that a translation is usually required as Swiss German and Standard German have mismatches on all linguistic levels. Unfortunately, there are not enough parallel text corpora available for training a proper translation system, nor enough in-domain speech translation (ST) data for training an ST system. We aim at investigating an end-to-end approach for multi-dialect Swiss German ST using transfer learning. Our ST model is based on an encoder-decoder architecture where we initialize the encoder with a cross-lingual speech representation model which is adapted to in-domain Swiss German speech data. We demonstrate that training the decoder on an out-of-domain ST corpus by preserving the encoder unit and then fine-tuning on in-domain ST data can be more effective than a cascade or vanilla direct ST.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130765186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

X-SHOT: Learning to Rank Voice Applications Via Cross-Locale Shard-Based Co-Training X-SHOT:学习通过跨区域基于分片的共同训练来排名语音应用程序

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688187

Zheng Gao, M. Abdelhady, Radhika Arava, Xibin Gao, Qian Hu, Wei Xiao, Thahir Mohamed

引用次数: 0

Warped Ensembles: A Novel Technique for Improving CTC Based End-to-End Speech Recognition 扭曲集成:一种改进基于CTC的端到端语音识别的新技术

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687953

K. Praveen, Hardik B. Sailor, Abhishek Pandey

{"title":"Warped Ensembles: A Novel Technique for Improving CTC Based End-to-End Speech Recognition","authors":"K. Praveen, Hardik B. Sailor, Abhishek Pandey","doi":"10.1109/ASRU51503.2021.9687953","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687953","url":null,"abstract":"Combining outputs from predictive models trained for similar tasks generally perform better than using a single model. Models trained on different domains could for example help in improving the performance with the help of information from complementary domains. The weights in the ensemble could also be adjusted to best suit the desired target domain. However, in End-to-End (E2E) based speech recognition solutions, this is not a simple task owing to the lack of time alignments in the outputs. This work presents a novel ensembling technique for E2E ASR systems trained using Connectionist Temporal Classification (CTC) loss, which uses a multi-sequence alignment technique for synchronising output time-frames across models, and then combines the outputs using weights obtained using a calibration technique introduced in our earlier work. Several experiments are conducted on multi-accent English data and the Librispeech corpus. Word Error Rates (WER) are evaluated and compared against the performance of ROVER (a system combination technique), and the best model in the system. When compared to the best model, average relative improvements of 6.46% and 8.94% are observed in unadapted in-domain and out-of-domain experiments respectively. Comparison to ROVER shows average relative improvements of 19.29% and 9.21% in in-domain and out-of-domain experiments.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116975411","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On-The-Fly Data Augmentation for Text-to-Speech Style Transfer 文本到语音风格转移的动态数据增强

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688074

Raymond Chung, B. Mak

{"title":"On-The-Fly Data Augmentation for Text-to-Speech Style Transfer","authors":"Raymond Chung, B. Mak","doi":"10.1109/ASRU51503.2021.9688074","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688074","url":null,"abstract":"Recent advanced text-to-speech (TTS) systems synthesize natural speeches. However, in many applications, it is desirable to synthesize utterances in a specific style. In this paper, we investigate synthesizing audios with three styles — news-casting, public speaking and storytelling — for a speaker who provides only neutral speech data. Firstly, considerable speech data were collected from the neutral speaker, and small amounts of speech from the wanted styles were collected from other speakers such that no speakers uttered in more than one style. All the data were used to train a basic multi-style multi-speaker TTS model. Secondly, augmented audios were created on-the-fly with the latest TTS model during its training and were used to further train the TTS model. Specifically, augmented data were created by ‘forcing’ a speaker to imitate stylish speeches of other three speakers by requiring their attention alignment matrices as similar as possible. Objective evaluation on the rhythm and pitch profile of the synthesized speech shows that the TTS model trained with our proposed data augmentation successfully transfers speech styles in these aspects. Subjective ABX evaluation also shows that stylish speeches synthesized by our proposed method are overwhelmingly preferred than those from a baseline TTS model by 40-60%.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115306745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Reconstructing Dual Learning for Neural Voice Conversion Using Relatively Few Samples 利用较少样本重构神经语音转换的对偶学习

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687965

Aolan Sun, Jianzong Wang, Ning Cheng, Methawee Tantrawenith, Zhiyong Wu, H. Meng, Edward Xiao, Jing Xiao

引用次数: 5

PSVD: Post-Training Compression of LSTM-Based RNN-T Models PSVD:基于lstm的RNN-T模型训练后压缩

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688220

Suwa Xu, Jinwon Lee, Jim Steele

{"title":"PSVD: Post-Training Compression of LSTM-Based RNN-T Models","authors":"Suwa Xu, Jinwon Lee, Jim Steele","doi":"10.1109/ASRU51503.2021.9688220","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688220","url":null,"abstract":"RNN-T has received a lot of attention recently since it achieves state-of-art WER in automatic speech recognition. To run the RNN-T model in real-time on resource-limited edge devices, model compression is often required. However, typical compression methods are still challenging to apply to RNN-T. First, it takes a lot of fine-tuning time and computing resources (e.g., up to several weeks even with multiple GPUs) due to the large size of training speech dataset and slow training speed of LSTMs. Second, the ASR training pipeline and datasets are often proprietary and not publicly available. Thus it is hard to fine-tune the pre-trained model after compression. In this paper, we propose PSVD (Post-training SVD) which could effectively improve the WER of a SVD com-pressed RNN-T model without requiring the large training dataset and costly back-propagation. Based on a small amount of test data, PSVD could quickly post-train a SVD compressed RNN-T model by leveraging alight-weight linear regression. In particular, we observed that multiple iterations of layer-sequential linear regression is effective in optimizing the compressed model. Compared to SVD compression with-out fine-tuning, PSVD could improve WER by up to 8.36% at a fraction of the compute time. To the best of our knowledge, our work is the first to leverage post-training compression for LSTM and RNN-T.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122837766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Comparative Study of Different Tokenization Strategies for Streaming End-to-End ASR 流端到端ASR中不同标记化策略的比较研究

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687921

Sachin Singh, Ashutosh Gupta, Aman Maghan, Dhananjaya N. Gowda, Shatrughan Singh, Chanwoo Kim

{"title":"Comparative Study of Different Tokenization Strategies for Streaming End-to-End ASR","authors":"Sachin Singh, Ashutosh Gupta, Aman Maghan, Dhananjaya N. Gowda, Shatrughan Singh, Chanwoo Kim","doi":"10.1109/ASRU51503.2021.9687921","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687921","url":null,"abstract":"Most End-to-End Automatic Speech Recognition (ASR) models use character-based vocabularies: characters, sub-words (BPE), or words. While these work well for training a monolingual model, there are certain limitations when ap-plying these to a multilingual model. Rare characters from character-rich languages like Korean can easily result in large vocabulary size, limiting the model's compactness. Repre-senting text at the level of bytes has also been proposed. However, a byte sequence representation of text is often much longer, which increases the decoding time and makes it computationally expensive for on-device use. Byte-based sub-words (BBPE) are proposed in neural machine translation for word representation but are still unexplored in the ASR domain. In this work, we conduct an empirical study comparing the above three tokenization strategies across three metrics: Word Error Rate (WER), model size, and the decoding time, which are critical for an on-device ASR. We did extensive experiments for both monolingual and bilingual, with languages belonging to same (English and Spanish) and different (English and Korean) language families. Our exper-iments show that BBPE and BPE models yield a similar WER for English and Spanish. While for a character-rich language like Korean, we get 26% and 14% relative WER improvement with BBPE monolingual and bilingual models, respectively. In contrast, the byte models trade-off small model size and a fixed vocabulary at the cost of high xRT. Among all three, we found the BBPE strategy to be the most flexible and optimal for most cases.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"775 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123282845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

An ASR N-Best Transcript Neural Ranking Model for Spoken Content Retrieval 语音内容检索的ASR N-Best转录物神经排序模型

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688208

Yasufumi Moriya, Gareth J. F. Jones

{"title":"An ASR N-Best Transcript Neural Ranking Model for Spoken Content Retrieval","authors":"Yasufumi Moriya, Gareth J. F. Jones","doi":"10.1109/ASRU51503.2021.9688208","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688208","url":null,"abstract":"Spoken Content Retrieval (SCR) using ASR transcripts is in-creasingly important for multimedia content archives. How-ever, SCR is often impacted by ASR errors. In recent years novel neural ranking methods for information retrieval (IR) have achieved improved search effectiveness over established methods. In this study, we examine neural ranking methods in SCR. Specifically we introduce two new neural ranking methods designed for use with errorful ASR transcripts. In the first, we train a neural ranking model using both man-ual transcripts and ASR transcripts or manual transcripts and artificially created ASR transcripts. In the second, we use an ASR N-best extension of two existing neural ranking methods: Deep Relevance Matching Model (DRMM) and Position-Aware Convolutional Recurrent Matching (PACRR). We report two sets of SCR experiments which evaluate our neural ranking methods. In the first, we use ASR transcripts to retrieve documents from an archive of spoken instruction videos. In the second, we examine a known-item search on a collection of user-generated spoken videos. We find that training the neural ranking model on both ASR and man-ual transcripts improves Mean Reciprocal Rank (MRR). The N-best extension of PACRR is particularly effective in both experiments in comparison to a standard BM25 IR model.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"112 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127563548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1