A Comparison of Several Word Clustering Models

Abstract

Sparse-data problem is a main issue that influences the performances of statistical language models; statistical language model based on word classes is an effective method to solve sparse-data problems. This paper presents a definition of word similarity by utilizing mutual information of adjoining words, and gives the definition of word set similarity based on word similarity, and puts forward a bottom-up hierarchical word clustering algorithm which can get global optimum. Experimental results show that the word clustering algorithm is of high executing speed and have good clustering performances. We then interpolated the class-based models with the word-based models and found that it mitigates remaining sparse-data problems of statistical language models.