一种用于优化增量爬虫性能的数据挖掘方法

Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003) Pub Date : 2003-10-13 DOI:10.1109/WI.2003.1241279

Hadrien Bullot, S. Gupta, M. Mohania

{"title":"一种用于优化增量爬虫性能的数据挖掘方法","authors":"Hadrien Bullot, S. Gupta, M. Mohania","doi":"10.1109/WI.2003.1241279","DOIUrl":null,"url":null,"abstract":"Crawlers visit the Web to maintain a local repository of Web pages up to date. We introduce another perspective to build an effective incremental crawler. Based on previous work in this field, we study how we can improve the performance of a crawler using data-mining. The information collected from the users can help the crawler to know which are the popular pages and to revisit them as soon as possible.","PeriodicalId":403574,"journal":{"name":"Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2003-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"18","resultStr":"{\"title\":\"A data-mining approach for optimizing performance of an incremental crawler\",\"authors\":\"Hadrien Bullot, S. Gupta, M. Mohania\",\"doi\":\"10.1109/WI.2003.1241279\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Crawlers visit the Web to maintain a local repository of Web pages up to date. We introduce another perspective to build an effective incremental crawler. Based on previous work in this field, we study how we can improve the performance of a crawler using data-mining. The information collected from the users can help the crawler to know which are the popular pages and to revisit them as soon as possible.\",\"PeriodicalId\":403574,\"journal\":{\"name\":\"Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003)\",\"volume\":\"38 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2003-10-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"18\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/WI.2003.1241279\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/WI.2003.1241279","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 18

摘要

爬虫访问Web以维护最新的Web页面的本地存储库。我们将引入另一种视角来构建有效的增量爬虫。在此基础上，我们研究了如何利用数据挖掘来提高爬虫的性能。从用户那里收集的信息可以帮助爬虫了解哪些是受欢迎的页面，并尽快重新访问它们。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

A data-mining approach for optimizing performance of an incremental crawler

Crawlers visit the Web to maintain a local repository of Web pages up to date. We introduce another perspective to build an effective incremental crawler. Based on previous work in this field, we study how we can improve the performance of a crawler using data-mining. The information collected from the users can help the crawler to know which are the popular pages and to revisit them as soon as possible.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003)

自引率

0.00%

发文量