麦克风阵列几何无关的多对讲机远程ASR: NTT系统用于DASR任务的CHiME-8挑战

IF 3.4 3区 计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Naoyuki Kamo , Naohiro Tawara , Atsushi Ando, Takatomo Kano, Hiroshi Sato, Rintaro Ikeshita, Takafumi Moriya, Shota Horiguchi, Kohei Matsuura, Atsunori Ogawa, Alexis Plaquet , Takanori Ashihara, Tsubasa Ochiai, Masato Mimura, Marc Delcroix, Tomohiro Nakatani, Taichi Asami, Shoko Araki
{"title":"麦克风阵列几何无关的多对讲机远程ASR: NTT系统用于DASR任务的CHiME-8挑战","authors":"Naoyuki Kamo ,&nbsp;Naohiro Tawara ,&nbsp;Atsushi Ando,&nbsp;Takatomo Kano,&nbsp;Hiroshi Sato,&nbsp;Rintaro Ikeshita,&nbsp;Takafumi Moriya,&nbsp;Shota Horiguchi,&nbsp;Kohei Matsuura,&nbsp;Atsunori Ogawa,&nbsp;Alexis Plaquet ,&nbsp;Takanori Ashihara,&nbsp;Tsubasa Ochiai,&nbsp;Masato Mimura,&nbsp;Marc Delcroix,&nbsp;Tomohiro Nakatani,&nbsp;Taichi Asami,&nbsp;Shoko Araki","doi":"10.1016/j.csl.2025.101820","DOIUrl":null,"url":null,"abstract":"<div><div>In this paper, we introduce a multi-talker distant automatic speech recognition (DASR) system we designed for the DASR task 1 of the CHiME-8 challenge. Our system performs speaker counting, diarization, and ASR. It handles a variety of recording conditions, from dinner parties to professional meetings and from two speakers to eight. We perform diarization first, followed by speech enhancement, and then ASR as the challenge baseline. However, we introduced several key refinements. First, we derived a powerful speaker diarization relying on end-to-end speaker diarization with vector clustering (EEND-VC), multi-channel speaker counting using enhanced embeddings from EEND-VC, and target-speaker voice activity detection (TS-VAD). For speech enhancement, we introduced a novel microphone selection rule to better select the most relevant microphones among those distributed microphones and investigated improvements to beamforming. Finally, for ASR, we developed several models exploiting Whisper and WavLM speech foundation models. In this paper, we present the original results we submitted to the challenge and updated results we obtained afterward. Our strongest system achieves a 63% relative macro tcpWER improvement over the baseline and outperforms the challenge best results on the NOTSOFAR-1 meeting evaluation data among geometry-independent systems.</div></div>","PeriodicalId":50638,"journal":{"name":"Computer Speech and Language","volume":"95 ","pages":"Article 101820"},"PeriodicalIF":3.4000,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Microphone array geometry-independent multi-talker distant ASR: NTT system for DASR task of the CHiME-8 challenge\",\"authors\":\"Naoyuki Kamo ,&nbsp;Naohiro Tawara ,&nbsp;Atsushi Ando,&nbsp;Takatomo Kano,&nbsp;Hiroshi Sato,&nbsp;Rintaro Ikeshita,&nbsp;Takafumi Moriya,&nbsp;Shota Horiguchi,&nbsp;Kohei Matsuura,&nbsp;Atsunori Ogawa,&nbsp;Alexis Plaquet ,&nbsp;Takanori Ashihara,&nbsp;Tsubasa Ochiai,&nbsp;Masato Mimura,&nbsp;Marc Delcroix,&nbsp;Tomohiro Nakatani,&nbsp;Taichi Asami,&nbsp;Shoko Araki\",\"doi\":\"10.1016/j.csl.2025.101820\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>In this paper, we introduce a multi-talker distant automatic speech recognition (DASR) system we designed for the DASR task 1 of the CHiME-8 challenge. Our system performs speaker counting, diarization, and ASR. It handles a variety of recording conditions, from dinner parties to professional meetings and from two speakers to eight. We perform diarization first, followed by speech enhancement, and then ASR as the challenge baseline. However, we introduced several key refinements. First, we derived a powerful speaker diarization relying on end-to-end speaker diarization with vector clustering (EEND-VC), multi-channel speaker counting using enhanced embeddings from EEND-VC, and target-speaker voice activity detection (TS-VAD). For speech enhancement, we introduced a novel microphone selection rule to better select the most relevant microphones among those distributed microphones and investigated improvements to beamforming. Finally, for ASR, we developed several models exploiting Whisper and WavLM speech foundation models. In this paper, we present the original results we submitted to the challenge and updated results we obtained afterward. Our strongest system achieves a 63% relative macro tcpWER improvement over the baseline and outperforms the challenge best results on the NOTSOFAR-1 meeting evaluation data among geometry-independent systems.</div></div>\",\"PeriodicalId\":50638,\"journal\":{\"name\":\"Computer Speech and Language\",\"volume\":\"95 \",\"pages\":\"Article 101820\"},\"PeriodicalIF\":3.4000,\"publicationDate\":\"2025-06-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computer Speech and Language\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0885230825000452\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Speech and Language","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0885230825000452","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

摘要

本文介绍了一种针对CHiME-8挑战赛中DASR任务1而设计的多话者远程自动语音识别(DASR)系统。我们的系统执行扬声器计数,拨号和ASR。它处理各种录音条件,从晚宴到专业会议,从两个发言者到八个发言者。我们首先进行拨号,然后是语音增强,然后是ASR作为挑战基线。然而,我们引入了几个关键的改进。首先,我们推导了基于端到端矢量聚类的扬声器拨号(EEND-VC),使用EEND-VC增强嵌入的多通道扬声器计数以及目标扬声器语音活动检测(TS-VAD)的强大扬声器拨号。在语音增强方面,我们引入了一种新的麦克风选择规则,以便在分布式麦克风中更好地选择最相关的麦克风,并研究了对波束形成的改进。最后,对于ASR,我们利用Whisper和WavLM语音基础模型开发了几个模型。在本文中,我们介绍了我们提交给挑战的原始结果和我们随后获得的更新结果。我们最强的系统在基线上实现了63%的相对宏观tcpWER改进,并且在几何无关的系统中优于NOTSOFAR-1会议评估数据的挑战最佳结果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Microphone array geometry-independent multi-talker distant ASR: NTT system for DASR task of the CHiME-8 challenge
In this paper, we introduce a multi-talker distant automatic speech recognition (DASR) system we designed for the DASR task 1 of the CHiME-8 challenge. Our system performs speaker counting, diarization, and ASR. It handles a variety of recording conditions, from dinner parties to professional meetings and from two speakers to eight. We perform diarization first, followed by speech enhancement, and then ASR as the challenge baseline. However, we introduced several key refinements. First, we derived a powerful speaker diarization relying on end-to-end speaker diarization with vector clustering (EEND-VC), multi-channel speaker counting using enhanced embeddings from EEND-VC, and target-speaker voice activity detection (TS-VAD). For speech enhancement, we introduced a novel microphone selection rule to better select the most relevant microphones among those distributed microphones and investigated improvements to beamforming. Finally, for ASR, we developed several models exploiting Whisper and WavLM speech foundation models. In this paper, we present the original results we submitted to the challenge and updated results we obtained afterward. Our strongest system achieves a 63% relative macro tcpWER improvement over the baseline and outperforms the challenge best results on the NOTSOFAR-1 meeting evaluation data among geometry-independent systems.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Computer Speech and Language
Computer Speech and Language 工程技术-计算机:人工智能
CiteScore
11.30
自引率
4.70%
发文量
80
审稿时长
22.9 weeks
期刊介绍: Computer Speech & Language publishes reports of original research related to the recognition, understanding, production, coding and mining of speech and language. The speech and language sciences have a long history, but it is only relatively recently that large-scale implementation of and experimentation with complex models of speech and language processing has become feasible. Such research is often carried out somewhat separately by practitioners of artificial intelligence, computer science, electronic engineering, information retrieval, linguistics, phonetics, or psychology.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信