Feedback-driven clustering for automated linking of web pages

Abstract

In this paper we propose and test a system that indexes a large collection of HTML documents (i.e. an entire web site) and automatically generates context-relevant inline text links between pairs of related documents (i.e. web pages). The goal of the system is threefold: to increase user interaction with the site being browsed, to discover relevant keywords for each document, and to effectively cluster the documents into semantically-significant groupings. The quality of the links is improved over time through passive user feedback collection. Our system can be deployed as a web service and has been tested on offline datasets as well as a live web site. A distinctive feature of our system is that it supports datasets that grow or change over time.