RLRAUC: Reinforcem ent learning based ranking algorithm using user clicks

Abstract

Because of great volume of web information, information retrieval process of a search engine is of great importance. For each query of user, the number of queries can reach hundred thousands, whereas a few number of the first results have the chance of being checked by user; therefore, a search engine pays attention to putting relevance results in the first ranks as a necessity. This paper introduces a reinforcement learning based ranking algorithm using user clicks, called RLRAUC, to put the relevance and favorite documents in the first ranks of query results. In the proposed algorithm, ranking system is the agent of learning system and selecting documents for displaying to user is considered as action. The reinforcement signal is calculated according to user click on documents. In this procedure, each pair of word-document in the user query is assigned a score according to the relevance of document. Documents in each repeating of learning would be sorted for next query based on changed scores and among these documents, according to document position in ranking list, random documents would be selected to be displayed to user. Learning process would be continued until it is converged to a stable ranking list. To evaluate proposed method, LETOR3 as well-known dataset has been used. Evaluation results indicate that RLRAUC is more effective than current ranking methods.