2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)最新文献

SI-Net: Multi-Scale Context-Aware Convolutional Block for Speaker Verification 基于上下文感知的多尺度卷积块说话人验证

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688119

Zhuo Li, Ce Fang, Runqiu Xiao, Wenchao Wang, Yonghong Yan

{"title":"SI-Net: Multi-Scale Context-Aware Convolutional Block for Speaker Verification","authors":"Zhuo Li, Ce Fang, Runqiu Xiao, Wenchao Wang, Yonghong Yan","doi":"10.1109/ASRU51503.2021.9688119","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688119","url":null,"abstract":"Utilizing multi-scale information adequately is essential for building a high-performance speaker verification (SV) system. Biological research shows that the human auditory system employs a multi-timescale processing mode to extract information and has a mechanism of integrating multi-scale information to encode sound information. Inspired by this, we propose a novel block, named Split-Integration (SI), to explore multi-scale context-aware feature learning at a granular level for speaker verification. Our model involves a pair of operations, (i) multi-scale split, which is designed to imitate the multi-timescale processing mode, extracting multi-scale features by grouping and stacking different sizes of filters, and (ii) dynamic integration, which aims at reflecting analogy with the fusion mechanism, introducing KL divergence to measure the complementarity between multi-scale features such that the model fully integrates multi-scale features and produces better speaker-discriminative representation. Experiments are conducted on Voxceleb and Speakers in the Wild(SITW) datasets. Results demonstrate that our approach achieves a relative 10%–20% improvement on equal error rate (EER) over a strong baseline in the SV task.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"302 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115615519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

DeepLip: A Benchmark for Deep Learning-Based Audio-Visual Lip Biometrics DeepLip:基于深度学习的视听嘴唇生物识别的基准

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688240

Meng Liu, Longbiao Wang, Kong-Aik Lee, Hanyi Zhang, Chang Zeng, J. Dang

{"title":"DeepLip: A Benchmark for Deep Learning-Based Audio-Visual Lip Biometrics","authors":"Meng Liu, Longbiao Wang, Kong-Aik Lee, Hanyi Zhang, Chang Zeng, J. Dang","doi":"10.1109/ASRU51503.2021.9688240","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688240","url":null,"abstract":"Audio-visual lip biometrics (AV-LB) has been an emerging biometrics technology that straddles auditory and visual speech processing. Previous works mainly focused on the front-end lip-based feature engineering combined with a shallow statistical back-end model. Over the past decade, convolutional neural network (CNN, or ConvNet) has been widely used and achieved good performance in computer vision and speech processing tasks. However, the lack of a sizeable public AV-LB database led to a stagnation in deep-learning exploration on AV-LB tasks. In addition to the dual audio-visual streams, one essential requirement on the video stream is the region of interest (ROI) around the lips has to be of sufficient resolution. To this end, we compile a moderate-size database using existing public databases. Using this database, we present a deep learning-based AV-LB benchmark, dubbed DeepLip11https://github.com/DanielMengLiu/DeepLip, realized with convolutional video and audio unimodal modules, and a multimodal fusion module. Our experiments show that DeepLip outperforms the traditional lip-biometrics system in context modeling and achieves over 50% relative improvements compared with its unimodal system, with an equal error rate of 0.75% and 1.11% on the test datasets, respectively.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131663885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Automatic Speech Recognition for Low-Resource Languages: The Thuee Systems for the IARPA Openasr20 Evaluation 低资源语言的自动语音识别:IARPA Openasr20评估的三种系统

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688260

Jing Zhao, Gui-Xin Shi, Guan-Bo Wang, Weiqiang Zhang

{"title":"Automatic Speech Recognition for Low-Resource Languages: The Thuee Systems for the IARPA Openasr20 Evaluation","authors":"Jing Zhao, Gui-Xin Shi, Guan-Bo Wang, Weiqiang Zhang","doi":"10.1109/ASRU51503.2021.9688260","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688260","url":null,"abstract":"The paper introduces our Automatic Speech Recognition (ASR) systems for the IARPA Open Automatic Speech Recognition Challenge (OpenASR20) as well as some post explorations with speech pre-training. We compete in the Constrained training condition for the 10 languages as team THUEE, under which the only speech data permissible for training of each language is a 10-hour corpus. We adopt the hybrid NN-HMM acoustic model and an N-gram Language Model (LM) to construct our basic ASR systems. The acoustic model is proposed as CNN-TDNNF-A, which combines Convolution Neural Network (CNN), Factored Time Delay Neural Network (TDNN-F) and self-attention mechanism. As for low-resource condition, we apply speed and volume perturbation, SpecAugment and reverberation for data enhancement as well as data clean-up to filter interference information. A series of pre-and-post-processing procedures for the evaluation set, such as Speech Activity Detection (SAD), system fusion and results filtering are carried out to obtain the final results. Furthermore, we exploit wav2vec 2.0 pre-trained model to obtain more effective speech representations for the hybrid system with a series of explorations, which brings about evident improvements.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127053648","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Far-Field Speech Recognition Based on Complex-Valued Neural Networks and Inter-Frame Similarity Difference Method 基于复值神经网络和帧间相似差法的远场语音识别

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688090

Y. Guo, Yifan Chen, Gaofeng Cheng, Pengyuan Zhang, Yonghong Yan

{"title":"Far-Field Speech Recognition Based on Complex-Valued Neural Networks and Inter-Frame Similarity Difference Method","authors":"Y. Guo, Yifan Chen, Gaofeng Cheng, Pengyuan Zhang, Yonghong Yan","doi":"10.1109/ASRU51503.2021.9688090","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688090","url":null,"abstract":"Far-field automatic speech recognition (ASR) is a challenging task due to the background noise and reverberation. To address this issue, we introduce a novel end-to-end multi-channel far-field ASR architecture. First, we use a complex-valued CNN based architecture designed for speech tasks as a neural beamformer. Second, we propose an auxiliary mod-ule called absolute position regression module (APRM) with a position prediction loss to help the neural beamformer be better aware of the corresponding frequencies of each input time-frequency (T-F) bin. Third, inspired by the short-term stationarity of human speech, we propose an approach called the Inter-Frame Similarity Difference (IFSD) method to au-tomatically select useful channels as the inputs of the ASR backend from the outputs of the neural beamformer. We also implement a complex-valued attention module for the output channels of the neural beamformer to utilize each other's in-formation, thereby preventing the final outputs from information loss. With the above innovations, our proposed model achieves 9.7% and 11.1% relative WER reductions over a DNN-MVDR baseline on the CHiME4 dataset and a dataset simulated using the Librispeech corpus.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125997046","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Towards Using Heterogeneous Relation Graphs for End-to-End TTS 异构关系图在端到端TTS中的应用

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9687876

Amrith Rajagopal Setlur, Aman Madaan, Tanmay Parekh, Yiming Yang, A. Black

{"title":"Towards Using Heterogeneous Relation Graphs for End-to-End TTS","authors":"Amrith Rajagopal Setlur, Aman Madaan, Tanmay Parekh, Yiming Yang, A. Black","doi":"10.1109/ASRU51503.2021.9687876","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9687876","url":null,"abstract":"Neural models for end-to-end text-to-speech (TTS) synthe-sis are increasingly outperforming traditional approaches in statistical parametric speech synthesis. Speech generation in these neural models predominantly relies on using free-form text as the input modality. However, the earlier statistical parametric models were built on encoded phonetic and syn-tactic features. In this work, we explore the possibility of explicitly feeding deterministic linguistic structure to a neural TTS system in the form of Heterogeneous Relational Graphs (HRGs), an expressive formalism capable of representing pho-netic and syntactic information. Specifically, we use Graph Convolutional Networks to learn structurally informed contin-uous representations of the HRGs, which can be seamlessly passed to the encoders of popular neural TTS models like TransformerTTS or Tacotron. Furthermore, our simple HRG based text-to-speech synthesis leverages the syntactic bias in HRGs as demonstrated by improvements in automated met-rics and human evaluation on i) the single speaker dataset LJSpeech; ii) the multi-speaker dataset Arctic; and iii) out-of-domain test sets from the Blizzard challenge. 11The code, trained models, and our dataset of HRGs will be released at https://github.com/ars22/GraphNeuralTTS/.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"05 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127254050","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

GLMSnet: Single Channel Speech Separation Framework in Noisy and Reverberant Environments 噪声和混响环境下的单通道语音分离框架

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688217

Huiyu Shi, Xi Chen, Tianlong Kong, S. Yin, P. Ouyang

引用次数: 1

3D Spatial Features for Multi-Channel Target Speech Separation 多通道目标语音分离的三维空间特征

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688198

Rongzhi Gu, Shi-Xiong Zhang, Meng Yu, Dong Yu

{"title":"3D Spatial Features for Multi-Channel Target Speech Separation","authors":"Rongzhi Gu, Shi-Xiong Zhang, Meng Yu, Dong Yu","doi":"10.1109/ASRU51503.2021.9688198","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688198","url":null,"abstract":"The use of speaker's directional information for speech sepa-ration and speech recognition has demonstrated the state-of-the-art performances on multi-talker scenarios. One major limitation of previous approaches using speaker's directional information is the significant performance degradation when the coming directions of two sound sources are close. To address these challenges, this paper proposed a set of new three-dimensional (3D) spatial features for target speech sep-aration, by leveraging all the 3D location information of the target speaker, including azimuth, elevation, and the distance to the microphone array center. Previous works in this area are extended in two important directions. First, the traditional 1D directional features are generalized to 3D spatial features. Thus more discriminative spatial diversity between speakers is achieved. Second, to unleash the full power of these 3D spatial features, a microphone pair-wise attention model is also proposed. The proposed features and models were evaluated on both simulated reverberant datasets and real recordings under near and far-field conditions. Exper-imental results show that both proposed 3D spatial features and attention models can significantly improve the separation performance as well as reducing the recognition error rate.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122689430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Voice to Action: Spoken Language Understanding for Memory-Constrained Systems 从声音到行动:内存受限系统的口语理解

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688150

Ashutosh Gupta, Aditya Jayasimha, Aman Maghan, Shatrughan Singh, Dhananjaya N. Gowda, Chanwoo Kim

{"title":"Voice to Action: Spoken Language Understanding for Memory-Constrained Systems","authors":"Ashutosh Gupta, Aditya Jayasimha, Aman Maghan, Shatrughan Singh, Dhananjaya N. Gowda, Chanwoo Kim","doi":"10.1109/ASRU51503.2021.9688150","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688150","url":null,"abstract":"Spoken Language Understanding (SLU) is the task of extracting semantic information from speech. Traditional architectures of SLU involve a two-stage pipeline consisting of an Automatic Speech Recognition (ASR) module that converts speech into text hypotheses, followed by a Natural Language Understanding (NLU) module that extracts semantic information from the text hypotheses. The increasing demand for light-weight modules, especially in on-device applications, promulgated end-to-end SLU, where a single module directly converts speech into semantic information. This work pro-poses a low resource, lightweight end-to-end SLU model that treats SLU as a sequence-to-sequence task and is optimized similar to the ASR task. The proposed model utilizes transfer learning from a heavy-resource, general domain ASR model to achieve an 10.8% absolute improvement in the Word Error Rate (WER) over the base model. We then utilize Knowledge Distillation to achieve a further 31.9% reduction in the model size with 22.7% relative WER improvement. In addition, we compress our models by more than 4 times using Low-Rank Approximation (LRA) method with minimum degradation in accuracy.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131552494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Short-Utterance Embedding Enhancement Method Based on Time Series Forecasting Technique for Text-Independent Speaker Verification 基于时间序列预测技术的短话语嵌入增强方法用于文本独立说话人验证

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688156

Jeonghwan Choi, Joon-Young Yang, Joon‐Hyuk Chang

{"title":"Short-Utterance Embedding Enhancement Method Based on Time Series Forecasting Technique for Text-Independent Speaker Verification","authors":"Jeonghwan Choi, Joon-Young Yang, Joon‐Hyuk Chang","doi":"10.1109/ASRU51503.2021.9688156","DOIUrl":"https://doi.org/10.1109/ASRU51503.2021.9688156","url":null,"abstract":"Short-utterance embedding, which is a speaker embedding extracted from a short utterance, shows poor speaker verification performance due to insufficient speaker information. To address the problem, we propose a method to map the set of short-utterance embeddings to a set of long-utterance embeddings based on a neural network. Specifically, a speech utterance is cropped into multiple segments whose durations are gradually increasing, and the speaker embeddings are extracted from the sequence of cropped segments using a pre-trained speaker embedding extractor. Subsequently, the sequence of embeddings is divided into a group of short-utterances embeddings and that of long-utterance embeddings. In our method, a sequence-to-sequence model based forecasting technique is employed, where an encoder transforms the group of short-utterance embeddings to a fixed-dimensional vector, and then a decoder converts the vector into a group of long-utterance embeddings. Experimental results on the VoxCeleb and Speakers in the Wild datasets show that our method improves the text-independent speaker verification performance under short utterance condition.","PeriodicalId":348315,"journal":{"name":"2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","volume":"395 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133644273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Analysis of Conversational Speech with Application to Voice Adaptation 会话语音分析及其在语音适应中的应用

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) Pub Date : 2021-12-13 DOI: 10.1109/ASRU51503.2021.9688146

Bhagyashree Mukherjee, A. Prakash, H. Murthy

引用次数: 3