Combining Syntactic Information with HMM for Term Extraction

Abstract

Aiming at the problem of Chinese thesaurus construction, we propose a method of using HMM to extract new terms from academic literature to expand automatically entry-words for Chinese thesaurus. This method converts the new terms extraction problem to a sequence labelling problem. It uses HMM fully integrated lexical information and syntactic information of new terms, as well as local context information, to learn automatically from the artificial corpus and obtain new terms extraction model. When new terms were extracted, Iturbi algorithm is used to extract automatically new terms from texts. Then this method receives these new terms as candidate entry-words. Eventually, we add content features filter conditions and frequency filter conditions for further selection. Experiment results show that the method has a good performance on terms extraction, and plays an important supporting role on expanding automatically entry-words for thesaurus.