Optimization of modular multi-speaker distant conversational speech recognition

Conducting multi-speaker distant conversational speech recognition on real meeting recordings is a challenging task and has recently become an active area of research. In this work, we focus on modular approaches to addressing this challenge, integrating continuous speech separation (CSS), automatic speech recognition (ASR), and speaker diarization in a pipeline. We explore the effective utilization of the high-performing separation model, TF-GridNet, within our system and propose integration techniques to enhance the performance of the ASR and diarization modules. Our system is evaluated on both the LibriCSS and the real-world CHiME-8 NOTSOFAR-1 dataset. Through a comprehensive analysis of the system’s generalization performance, we identify key areas for further improvement in the front-end module.