Topic segmentation of news speech using word similarity

Abstract

Conventional topic segmentation utilizes cosine measure as the similarity between consecutive passages. However, the cosine measure has a problem that it can not reflect the similarity unless exactly the same words are included in the passages. To solve this problem, in this paper, we propose a method to acquire the word similarity between different words from the input data directly and automatically by managing to collect the same topic sections. Further more, we propose a method to compute the passage similarity based on the word similarity. Finally we propose a method of topic segmentation based on the passage similarity in an unsupervised mode.