Retriever: a self-training agent for intelligent information discovery

Abstract

With the exponential growth of the Internet and the volume of information published on it, searching for information of interest has become a very difficult and time-consuming task. In this paper, we present 'Retriever', an autonomous agent that executes user queries and returns high-quality results to the user. Retriever utilizes existing search engines to obtain the starting points for its subsequent autonomous exploration of the Web. Then it conducts a self-training process in order to learn the query domain and to increase its efficiency. When the query domain is learned, the agent expands the original query, reforms its search strategy and goes out looking for the documents to be presented to the user. It also incorporates relevance feedback in order to perform subsequent searches on the same query.