{"title":"Cross-Stitch Network Based System for Sound Event Localization and Detection in L3DAS22 Challenge","authors":"Jisheng Bai, Siwei Huang, Yafei Jia, Mou Wang, Jianfeng Chen","doi":"10.21437/l3das.2022-2","DOIUrl":"https://doi.org/10.21437/l3das.2022-2","url":null,"abstract":"Sound event localization and detection (SELD) has great po-tential importance in daily life. Joint training of SELD can simultaneously share and model the acoustic knowledge of sound event detection and source localization. In this paper, we propose a novel SELD system based on a two-branch cross-stitch neural network. First, the proposed neural network takes 4-channel log-Mel energies and 3-channel intensity vector as two-branch inputs. Then we incorporate the cross-stitch unit and Transformer encoder to share and model the acoustic representations for SELD. Besides, we present a time-domain data augmentation method to effectively improve the performance of SELD. We evaluated the proposed system on the dataset of ICASSP 2022 L3DAS22 Challenge Task 2. Results show that our system outperforms the official baseline system by a large margin. We employ an ensemble of several models and achieve further improvement in the evaluation metrics.","PeriodicalId":412193,"journal":{"name":"L3DAS22: Machine Learning for 3D Audio Signal Processing","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132620954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Heitor R. Guimarães, Wesley Beccaro, M. A. Ramírez
{"title":"A Perceptual Loss Based Complex Neural Beamforming for Ambix 3D Speech Enhancement","authors":"Heitor R. Guimarães, Wesley Beccaro, M. A. Ramírez","doi":"10.21437/l3das.2022-4","DOIUrl":"https://doi.org/10.21437/l3das.2022-4","url":null,"abstract":"This work proposes a novel approach to B-Format AmbiX 3D speech enhancement based on the short-time Fourier transform (STFT) representation. The model is a Fully Complex Convolutional Network (FC2N) that estimates a mask to be applied to the input features. Then, a final layer is responsible for converting the B-format to a monaural representation in which we apply the inverse STFT (ISTFT) operation. For the optimization process, we use a compounded loss function, applied in the time-domain, based on the short-time objective intelligibility (STOI) metric combined with a perceptual loss on top of the wav2vec 2.0 model. The approach is applied on Task 1 of the L3DAS22 challenge, where our model achieves a score of 0.845 in the metric proposed by the challenge, using a subset of the development set as reference.","PeriodicalId":412193,"journal":{"name":"L3DAS22: Machine Learning for 3D Audio Signal Processing","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123980842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Cross-Stitch Network with Adaptive Loss Weightage for Sound Event Localization and Detection","authors":"T. K. Chan, R. Das","doi":"10.21437/l3das.2022-3","DOIUrl":"https://doi.org/10.21437/l3das.2022-3","url":null,"abstract":"A Sound Event Localization and Detection system is capable of identifying the type and source of an acoustic event in a 3-dimensional space. Typically, such a system is trained using a Multi-Task Learning (MTL) framework, where the loss propagated is a linear combination of individual task losses. However, it has been found that the hard-parameter sharing strategy for an MTL framework can degrade the system performance. In addition, deriving the optimal loss combination can be time-consuming empirically and may not be the best way. This work proposes a cross-stitch network with a novel attention module that improves the feature representations. Further, we propose the use of a loss balancing algorithm to weigh the loss contribution adaptively, thereby eliminating the need to tune the loss weightage empirically. The proposed system is then evaluated on L3DAS22 challenge dataset as a part of our challenge participation and achieves a significant performance improvement of over 20% compared to the state-of-the-art SELDnet. We also note that our system ranked 3 rd in the L3DAS22 challenge Task 2 without any data augmentation or external dataset to increase the training samples.","PeriodicalId":412193,"journal":{"name":"L3DAS22: Machine Learning for 3D Audio Signal Processing","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132648059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"L3DAS22: Exploring Loss Functions for 3D Speech Enhancement","authors":"Tyler Vuong, Mark Lindsey, Yangyang Xia, R. Stern","doi":"10.21437/l3das.2022-1","DOIUrl":"https://doi.org/10.21437/l3das.2022-1","url":null,"abstract":"This work explores the effects of different speech enhancement loss functions traditionally used for monophonic signals when applied to the L3DAS22 Challenge 3D Speech Enhancement Task. In addition to baseline time-domain losses, loss functions in the time-frequency and modulation domains are introduced to a common network. These losses are compared by their effect on system performance for the task, and then by their correlation with important speech enhancement metrics, such as word error rate (WER) and short-time objective intelligibility (STOI). Findings show that the Phase-Constrained Magnitude (PCM) loss paired with modulation loss improved performance by 12 . 1% relative to the L3DAS22 baseline in terms of the challenge’s evaluation metric. It was also found that the the modulation distance is consistently more correlated with WER and STOI than other metrics.","PeriodicalId":412193,"journal":{"name":"L3DAS22: Machine Learning for 3D Audio Signal Processing","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127879879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}