A speech emotion recognition method based on DST-GDCC and text-to-speech data augmentation

Abstract

Speech Emotion Recognition (SER) is a critical component of human-machine interaction, yet it confronts two fundamental challenges: limited feature extraction capabilities and data scarcity. This paper proposes a unified framework that synergistically addresses both issues through the co-design of a novel SER model and a high-quality data augmentation strategy. At its core, the Deformable Speech Transformer (DST) and the Gated Dilation Causal Convolution (GDCC) are introduced, which are combined to form the DST-GDCC model for superior feature extraction. The DST component adaptively captures multi-granular acoustic features, while the GDCC module explicitly models the spatiotemporal causality of speech emotions. However, the full potential of such an advanced model is often constrained by scarce training data. To overcome this limitation, a Text-to-Speech (TTS) data augmentation method is incorporated, leveraging a pre-trained GPT-SoVITS model to synthesize high-fidelity, emotion-conditioned speech samples. Crucially, these two components form a virtuous cycle: the powerful discriminative ability of the DST-GDCC model is leveraged in a dual-stage screening mechanism to ensure the quality of the synthetic data, while the expanded, high-quality dataset, in turn, enables the model to realize its full potential. Experimental results demonstrate the framework's effectiveness. The DST-GDCC model itself achieves significant accuracy improvements over baselines (2.66% on IEMOCAP, 5.02% on MELD, 5.83% on CASIA). More importantly, the synergistic integration with TTS data augmentation yields further gains of 3.13% on IEMOCAP and 3.33% on CASIA, validating the framework's capability to systematically elevate SER performance.