2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)最新文献_第4页

Are You Dictating to Me? Detecting Embedded Dictations in Doctor-Patient Conversations 你在对我发号施令吗?检测医患对话中的嵌入式口述

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688118

Thomas Schaaf, Longxiang Zhang, Alireza Bayestehtashk, Mark C. Fuhs, Shahid Durrani, Susanne Burger, M. Woszczyna, T. Polzin

{"title":"Are You Dictating to Me? Detecting Embedded Dictations in Doctor-Patient Conversations","authors":"Thomas Schaaf, Longxiang Zhang, Alireza Bayestehtashk, Mark C. Fuhs, Shahid Durrani, Susanne Burger, M. Woszczyna, T. Polzin","doi":"10.1109/ASRU51503.2021.9688118","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688118","url":null,"abstract":"Medical scribes chart doctor-patient conversations in real time or by listening to an audio recording afterwards. Doctors sometimes dictate during a patient encounter, a highly informative part for a scribe. We introduced a light-weight annotation schema and ana-lyzed recordings of 105 randomly selected doctor-patient encounters from 21 physicians to quantify the frequency and automatically de-tect dictated regions. Dictation behavior of individual doctors was consistent but varied among them. A linguistic analysis is provided to describe differences of doctors speech when talking to a patient or dictating. A description of the data is given, highlighting challenges of segmenting audio into conversation and dictation regions. We in-vestigate different features and methods to segment conversations including keyword spotting, acoustic features and class-conditioned language models. Results are anchored to a majority class base-line. Using only acoustic features allows to predict dictated speech without the need of a speech recognition system performing com-parable to a rule-based approach using lexical features derived from a speech recognition system. Performance is assessed using leave-one-physician-out cross validation and an analysis using a random forest classifier indicates that language model derived features are most useful, and that a combination of acoustic and lexical features performed best.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"106 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121393918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An Evaluation Benchmark for Automatic Speech Recognition of German-English Code-Switching 德语-英语语码转换语音自动识别的评价基准

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687941

Abbas Khosravani, Philip N. Garner, Alexandros Lararidis

引用次数: 2

Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understanding 多回合端到端口语理解的细心语境结转

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688079

Kai Wei, Thanh Tran, Feng-Ju Chang, Kanthashree Mysore Sathyendra, Thejaswi Muniyappa, Jing Liu, A. Raju, Ross McGowan, Nathan Susanj, A. Rastrow, Grant P. Strimel

{"title":"Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understanding","authors":"Kai Wei, Thanh Tran, Feng-Ju Chang, Kanthashree Mysore Sathyendra, Thejaswi Muniyappa, Jing Liu, A. Raju, Ross McGowan, Nathan Susanj, A. Rastrow, Grant P. Strimel","doi":"10.1109/ASRU51503.2021.9688079","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688079","url":null,"abstract":"Recent years have seen significant advances in end-to-end (E2E) spoken language understanding (SLU) systems, which directly predict intents and slots from spoken audio. While dialogue history has been exploited to improve conventional text-based natural language understanding systems, current E2E SLU approaches have not yet incorporated such critical contextual signals in multi-turn and task-oriented dialogues. In this work, we propose a contextual E2E SLU model ar-chitecture that uses a multi-head attention mechanism over encoded previous utterances and dialogue acts (actions taken by the voice assistant) of a multi-turn dialogue. We detail alternative methods to integrate these contexts into the state-of-the-art recurrent and transformer-based models. When applied to a large de-identified dataset of utterances collected by a voice assistant, our method reduces average word and seman-tic error rates by 10.8% and 12.6%, respectively. We also present results on a publicly available dataset and show that our method significantly improves performance over a non-contextual baseline.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124535806","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

An End-to-End Far-Field Keyword Spotting System with Neural Beamforming 端到端神经波束形成远场关键词定位系统

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688326

Xuan Ji, Lu Lu, Fuming Fang, Jianbo Ma, Lei Zhu, Jinke Li, Dongdi Zhao, Meilin Liu, Feijun Jiang

引用次数: 1

Attention Based Model for Segmental Pronunciation Error Detection 基于注意的分词发音错误检测模型

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687993

Jose Antonio Lopez Saenz, Md. Asif Jalal, R. Milner, Thomas Hain

引用次数: 1

Detecting Emotion Carriers by Combining Acoustic and Lexical Representations 结合声学和词汇表征检测情感载体

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687893

Sebastian P. Bayerl, Aniruddha Tammewar, K. Riedhammer, G. Riccardi

{"title":"Detecting Emotion Carriers by Combining Acoustic and Lexical Representations","authors":"Sebastian P. Bayerl, Aniruddha Tammewar, K. Riedhammer, G. Riccardi","doi":"10.1109/ASRU51503.2021.9687893","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687893","url":null,"abstract":"Personal narratives (PN) – spoken or written – are recollections of facts, people, events, and thoughts from one's own experience. Emotion recognition and sentiment analysis tasks are usually defined at the utterance or document level. However, in this work, we focus on Emotion Carriers (EC) defined as the segments (speech or text) that best explain the emotional state of the narrator (“loss of father”, “made me choose”). Once extracted, such EC can provide a richer representation of the user state to improve natural language understanding and dialogue modeling. In previous work, it has been shown that EC can be identified using lexical features. However, spoken narratives should provide a richer description of the context and the users' emotional state. In this paper, we leverage word-based acoustic and textual embeddings as well as early and late fusion techniques for the detection of ECs in spoken narratives. For the acoustic word-level representations, we use Residual Neural Networks (ResNet) pretrained on separate speech emotion corpora and fine-tuned to detect EC. Experiments with different fusion and system combination strategies show that late fusion leads to significant improvements for this task.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121571093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Robust Speech-Age Estimation Using Local Maximum Mean Discrepancy Under Mismatched Recording Conditions 不匹配记录条件下基于局部最大均值差异的鲁棒语音年龄估计

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687861

Naohiro Tawara, A. Ogawa, Yuki Kitagishi, Hosana Kamiyama, Yusuke Ijima

{"title":"Robust Speech-Age Estimation Using Local Maximum Mean Discrepancy Under Mismatched Recording Conditions","authors":"Naohiro Tawara, A. Ogawa, Yuki Kitagishi, Hosana Kamiyama, Yusuke Ijima","doi":"10.1109/ASRU51503.2021.9687861","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687861","url":null,"abstract":"A recently proposed time-delay neural network (TDNN)-based age estimation system has yielded state-of-the-art performance in speech-age estimation tasks. However, the performance of this TDNN-based system can seriously degrade when the recording conditions of each utterance are different in the training and testing phases. To tackle this problem, we examine the efficiencies of a series of unsupervised domain adaptation (UDA) methods to obtain the model invariance against the difference of these conditions. In particular, we propose using local maximum mean discrepancy (LMMD) with soft-target labels to consider an ordinal relationship between age labels. In most UDA methods, the model is trained to obtain domain invariant representations by minimizing the statistical difference of the distributions between labeled source and unlabeled target data without considering their age class labels. In contrast, our LMMD-based approach locally minimizes the differences in their distributions on each age class while considering adjacent age classes using soft-target labels. We conducted speech-age estimation experiments on in-house datasets under mismatched conditions including different background noise, reverberation, and microphones. The experimental comparison demonstrated that the LMMD-based method contributed to efficiently reducing the effect of mismatches of input data, yielding significant improvements over other UDA methods, such as MMD and reverse gradients.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"103 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126633460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Improving HS-DACS Based Streaming Transformer ASR with Deep Reinforcement Learning 用深度强化学习改进基于HS-DACS的流变压器ASR

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688268

Mohan Li, R. Doddipatla

{"title":"Improving HS-DACS Based Streaming Transformer ASR with Deep Reinforcement Learning","authors":"Mohan Li, R. Doddipatla","doi":"10.1109/ASRU51503.2021.9688268","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688268","url":null,"abstract":"Transformer-based systems, though have achieved state-of-the-art performance on a wide range of automatic speech recognition (ASR) tasks, are subject to severe latency issues during inference that limit their deployment in real world applications. To enable online decoding, we recently proposed Decoder-end adaptive computation steps (DACS) and the head-synchronous version (HS-DACS) algorithms, which were shown to reduce the computational cost for decoding and close the gap in performance between offline and streaming Transformer ASR. In DACS/HS-DACS based systems, the halting position that triggers the ASR output was determined with an accumulation threshold that was set arbitrarily. In this paper, we propose a deep reinforcement learning approach to improve HS-DACS where an external agent is utilised to optimise the halting position. Experiments on AIShell-1, Tedlium-2 and Librispeech datasets show that the proposed method can further cut down the computation cost in inference with relative gains of 40.0%, 32.8% and 53.4% respectively, and still maintain similar ASR performance when compared with HS-DACS.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129294458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Speech Emotion Recognition Using Semi-Supervised Learning with Efficient Labeling Strategies 基于高效标注策略的半监督学习语音情感识别

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687938

Zhi Zhu, Yoshinao Sato

引用次数: 2

Learning Language and Speaker Information for Code-Switch Speech Synthesis with Limited Data 有限数据下转换码语音合成的语言学习和说话人信息

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687961

Ms Chai, Shaotong Guo, Cheng Gong, Longbiao Wang, J. Dang, Ju Zhang

{"title":"Learning Language and Speaker Information for Code-Switch Speech Synthesis with Limited Data","authors":"Ms Chai, Shaotong Guo, Cheng Gong, Longbiao Wang, J. Dang, Ju Zhang","doi":"10.1109/ASRU51503.2021.9687961","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687961","url":null,"abstract":"End-to-end speech synthesis demonstrates remarkable performance in monolingual speech, whereas code-switching (CS) speech synthesis remains a challenge owing to the sparsity of data and diverse syntactic structures across languages. Previous studies show that large mixed-lingual corpora are essential for effective learning text/language representations and target speaker information. In this study, we propose a method using three independent encoders (text, language, and speaker), which requires only a small amount of mixed-lingual data to realize the CS speech synthesis of Mandarin and English. Additionally, to distinguish between Mandarin and English, we investigate two text-representation methods: (1) the implicit method, which uses Pinyin and the CMU11http://www.speech.cs.cmu.edu/cgi-bin/cmudict dictionary to represent both languages; and (2) the explicit method, which uses language markers i.e., masks, to differentiate the languages. Through our proposed method, we can improve synthesized speech in terms of quality and speaker similarity using a small amount of mixed-lingual data. In addition, the experimental results demonstrate that the proposed method achieves performance improvement of 0.06 in terms of the mean opinion score and absolute improvement of 0.64% in terms of the character error rate compared to the baseline method.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129563154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0