Dynamically Jointing character and word embedding for Chinese text Classification

Abstract

Chinese text classification is drawing attention in these few years. Different from English texts, there is no natural separator between Chinese words. With the development of deep learning, many character-level only models have been proposed for Chinese text classification to tackle this problem, which have achieved more success than word-level models. But the word information is also important for Chinese text representation, especially for short texts with less information. However, most of neural network models either just concatenate character-level representation and word-level representation, or use massive external knowledge to represent the whole text, which is complex and time-consuming. For better and easier representing the Chinese text without any external knowledge and using as much character and word information as possible, we propose a simple model jointed character and word embedding dynamically, called DJCW. Firstly, the character-level and word-level BiLSTM Model is introduced to extract features of texts with indefinite lengths. Secondly, the char and word are weightedly combined and the weights are changed dynamically. Finally, experiments conducted on five open-source text datasets show our model can handle the texts with different lengths and has achieved good stability results.