A Self-Organizing Search Engine for RSS Syndicated Web Contents

Abstract

The exponentially growing information published on the Web relies largely on a few major search engines like Google to be brought to the public nowadays. This raises issues such as: 1. how many percents of coverage do these search engines provide for the whole shared contents over the Internet? 2. how easy is it to find less popular contents from the Web through the page ranking system of these search engines? In fact, the increasing dynamics of the information distributed on the Internet challenge the flexibility of these centralized search engines. With the amount of structured and semi-structured data increase on the Internet, self-organizing search engines that are capable of providing sufficient coverage for data that follow certain structures get more and more attractive. In this paper, we propose a self-organizing search engine soSpace for RSS syndicated web data. soSpace is built on structured peer-to-peer technology. It enables indexing and searching of frequently updated web information described by RSS feed. Our experiment results show that it has good scalability as the contents increase. The recall and precision rate of the result are satisfactory as well.