DEEP-CWS: Distilling Efficient pre-trained models with Early exit and Pruning for scalable Chinese Word Segmentation

Abstract

Chinese Word Segmentation (CWS) is essential for a broad spectrum of tasks in natural language processing (NLP). However, the high inference cost of large pre-trained models like BERT and RoBERTa restricts their scalability in practical deployments. To overcome this limitation, we introduce DEEP-CWS, a novel approach for efficient CWS that distills pre-trained transformer models into lightweight CNNs, incorporating pruning, early exit mechanisms, and ONNX optimization to improve inference speed significantly. Our method achieves over 100 times speedup in inference latency relative to the teacher model without compromising segmentation quality, with an F1 score of 97.81 on the PKU benchmark. These characteristics make DEEP-CWS particularly well-suited for real-time scenarios and large-scale processing. Extensive experiments on public benchmarks and a legal-domain dataset validate the robustness and transferability of our framework. We also release our code base to support reproducibility and future research.