Multi-Scale Residual Attention Network for Chinese Speech Recognition Through Collaborative Design.

Abstract

In recent years, the application of multi-scale residuals and attention mechanisms in automatic speech recognition has made some progress. However, existing speech recognition systems still have critical limitations. Many existing systems fail to adequately capture multi-scale speech features, struggling with representation gaps and often relying on inefficient serial attention mechanisms. Additionally, lightweight models, while efficient, often isolate information within grouped architectures, limiting the global feature interaction necessary for speech recognition. To address these persistent challenges, we introduce MSRDA-GSCA, an architecture that synergistically integrates a Multi-Scale Residual Deep Convolutional Attention (MSRDA) network with a Group Shuffled Co-Attention (GSCA) mechanism. MSRDA uses parallel convolutions with varied kernels and dilations, plus adaptive residual connections, to capture diverse temporal dependencies and align heterogeneous features. Its dual-path attention recalibrates channel and spatial information with element-wise max fusion. GSCA employs sequential position-wise channel and spatial attention with intermediate channel shuffling, enabling cross-group interaction and enhancing global feature synergy. Experiments on THCHS-30 and ST-CMDS datasets show that the proposed model outperforms existing methods in Character Error Rate (CER) and Word Error Rate (WER). Ablation studies confirm the complementarity of MSRDA and GSCA. With only 4.65 M parameters, MSRDA-GSCA achieves high recognition accuracy while maintaining low computational complexity for efficient deployment.