Multi-level topic detection algorithm for Netnews Specials

Abstract

This paper investigates the topic detection method in Netnews Specials Detection (NSD). We found that when the traditional clustering algorithms are used in NSD, the same topic is usually split into several pieces and the result is not satisfying. So a new algorithm is proposed which uses a multi-level model, better suited for NSD. Firstly, such algorithm elevates the accuracy of single-layer clustering by introducing hot search words, a selective dictionary, and an advanced weight formula. Secondly, the multiple-level model not only avoids the problem of topic over-split but also establishes a structure for Netnews Specials, which lays the foundation for quick viewing, positioning and retrieval. Experimental results show that the algorithm in the real test corpus have high accuracy, doing a better job than the traditional clustering method.