使用基于文本的过滤器抓取Web论坛

2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI) Pub Date : 2017-09-01 DOI:10.1109/ICPCSI.2017.8392037

Priyanka Bandagale, Atiya R. Kazi

{"title":"使用基于文本的过滤器抓取Web论坛","authors":"Priyanka Bandagale, Atiya R. Kazi","doi":"10.1109/ICPCSI.2017.8392037","DOIUrl":null,"url":null,"abstract":"Forum websites contain very valuable information about frequently occurred problems in any subject. The good thing about these forums is, it is organized in a very proper format so that people can navigate easily to their expected discussion page. Now a days search engines are becoming very specific for different types of websites. So, they are making special crawlers for different types of websites to crawl exact information from raw html code. The research on forum crawlers is increased since last few years. But the main challenge in forum crawlers is finding similarity between URLs from different forum sites. This paper focuses on crawling forum URLs and classifying them as Entry, Index, Thread and Page Flipping URLs.","PeriodicalId":6589,"journal":{"name":"2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI)","volume":"1 1","pages":"1856-1859"},"PeriodicalIF":0.0000,"publicationDate":"2017-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Web forum crawling using text-based filters\",\"authors\":\"Priyanka Bandagale, Atiya R. Kazi\",\"doi\":\"10.1109/ICPCSI.2017.8392037\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Forum websites contain very valuable information about frequently occurred problems in any subject. The good thing about these forums is, it is organized in a very proper format so that people can navigate easily to their expected discussion page. Now a days search engines are becoming very specific for different types of websites. So, they are making special crawlers for different types of websites to crawl exact information from raw html code. The research on forum crawlers is increased since last few years. But the main challenge in forum crawlers is finding similarity between URLs from different forum sites. This paper focuses on crawling forum URLs and classifying them as Entry, Index, Thread and Page Flipping URLs.\",\"PeriodicalId\":6589,\"journal\":{\"name\":\"2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI)\",\"volume\":\"1 1\",\"pages\":\"1856-1859\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2017-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICPCSI.2017.8392037\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICPCSI.2017.8392037","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

摘要

论坛网站包含关于任何主题中经常出现的问题的非常有价值的信息。这些论坛的好处是，它以非常合适的格式组织，以便人们可以轻松地导航到他们期望的讨论页面。现在，搜索引擎对不同类型的网站变得非常具体。因此，他们正在为不同类型的网站制作特殊的抓取工具，以便从原始html代码中抓取准确的信息。近年来，论坛爬虫的研究越来越多。但是论坛爬虫的主要挑战是找到来自不同论坛站点的url之间的相似性。本文主要对论坛url进行抓取，并将其分类为入口url、索引url、线程url和翻页url。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Web forum crawling using text-based filters

Forum websites contain very valuable information about frequently occurred problems in any subject. The good thing about these forums is, it is organized in a very proper format so that people can navigate easily to their expected discussion page. Now a days search engines are becoming very specific for different types of websites. So, they are making special crawlers for different types of websites to crawl exact information from raw html code. The research on forum crawlers is increased since last few years. But the main challenge in forum crawlers is finding similarity between URLs from different forum sites. This paper focuses on crawling forum URLs and classifying them as Entry, Index, Thread and Page Flipping URLs.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI)

自引率

0.00%

发文量