A Deep Attention Network for Chinese Word Segment

Abstract

Character-level sequence label tagging is the most efficient way to solve unknown words problem for Chinese word segment. But the most widely used model, Conditional Random Fields (CRF), needs a large amount of manual design features. So it is appropriate to combine CRF and neural networks such as recurrent neural network (RNN), which is adopted in many natural language processing (NLP) tasks. However, RNN is rather slow because of the timing dependence between computations and not good at capturing local information of the sentence. In order to solve this problem, we introduce a self-attention mechanism, which completes the calculation between the different positions of the sentence with the same distance, into CWS. And we propose a deep neural network, which combines convolution neural networks and self-attention mechanism. Then, we evaluate the model on the PKU dataset and the MSR dataset. The results show that our model perform much better.