基于DTB的智能主题网络爬虫

2010 International Conference on Web Information Systems and Mining Pub Date : 2010-10-23 DOI:10.1109/WISM.2010.155

Ming-sheng Zhao, Peng Zhu, Tianchi He

{"title":"基于DTB的智能主题网络爬虫","authors":"Ming-sheng Zhao, Peng Zhu, Tianchi He","doi":"10.1109/WISM.2010.155","DOIUrl":null,"url":null,"abstract":"Web crawling is a fundamental step in many Web applications, such as search engine and data mining. This paper proposes an intelligent topic Web crawler based on DTB (dynamic topic base), which through studying on Web crawlers which filter URLs based on different methods. This Web crawler can update the topic base automatically and improve the accuracy of URL filtering. Experimental results show that the proposed Web crawler can fetch more topic relevant Web pages by crawling less Web space and in less time.","PeriodicalId":119569,"journal":{"name":"2010 International Conference on Web Information Systems and Mining","volume":"21 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2010-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"6","resultStr":"{\"title\":\"An Intelligent Topic Web Crawler Based on DTB\",\"authors\":\"Ming-sheng Zhao, Peng Zhu, Tianchi He\",\"doi\":\"10.1109/WISM.2010.155\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Web crawling is a fundamental step in many Web applications, such as search engine and data mining. This paper proposes an intelligent topic Web crawler based on DTB (dynamic topic base), which through studying on Web crawlers which filter URLs based on different methods. This Web crawler can update the topic base automatically and improve the accuracy of URL filtering. Experimental results show that the proposed Web crawler can fetch more topic relevant Web pages by crawling less Web space and in less time.\",\"PeriodicalId\":119569,\"journal\":{\"name\":\"2010 International Conference on Web Information Systems and Mining\",\"volume\":\"21 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2010-10-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"6\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2010 International Conference on Web Information Systems and Mining\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/WISM.2010.155\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2010 International Conference on Web Information Systems and Mining","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/WISM.2010.155","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 6

摘要

Web爬行是许多Web应用程序(如搜索引擎和数据挖掘)中的一个基本步骤。本文通过对基于不同方法过滤url的网络爬虫的研究，提出了一种基于动态主题库的智能主题网络爬虫。该网络爬虫可以自动更新主题库，提高URL过滤的准确性。实验结果表明，该网络爬虫可以在更短的时间内以更小的网络空间抓取到更多与主题相关的网页。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

An Intelligent Topic Web Crawler Based on DTB

Web crawling is a fundamental step in many Web applications, such as search engine and data mining. This paper proposes an intelligent topic Web crawler based on DTB (dynamic topic base), which through studying on Web crawlers which filter URLs based on different methods. This Web crawler can update the topic base automatically and improve the accuracy of URL filtering. Experimental results show that the proposed Web crawler can fetch more topic relevant Web pages by crawling less Web space and in less time.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2010 International Conference on Web Information Systems and Mining

自引率

0.00%

发文量