Similarity Measures for Chinese Short Text Based on Representation Learning

Abstract

Similarity measure in Chinese short text is an important prerequisite for many content-based texts or documents retrieval tasks. In this paper, we propose a fast method for representing Chinese short texts to calculate the similarity between texts. The method is based on the representation of Chinese words. First, Chinese word representation is learned by a deep neural network with local context embedding and global context. Then, the words in short text are replaced by the learned representations of Chinese words and the short text is represented by dynamic average-weighted function depending on target text. Next, the cosine similarity method is used for the similarity measurement between texts. Last, experiment shows the semantic by visualizing the result of Chinese word representation learning and the experiment on similarity measure demonstrates the effectiveness of our short text representation method.