Similarity Computation of Web Pages of Focused Crawler

Abstract

Due to the dynamic nature of the Web, it becomes harder to find relevant and recent information. More and more people begin to use focused crawler to get information in their special fields today. However, the Similarity Computation based text is incompetent, because the page consists of not only text but also multimedia contents, such as image, audio, video and so on. In the field of the focused crawler the page structure plays a key role in the similarity computation too. In this paper we introduce a new method to have similarity computation according the page structure and content which can make web page similarity computation exactly and crawling efficiently which will bring benefits for Web analysis and get information easily for users.