Learning the Latent Semantic Space for Ranking in Text Retrieval

Abstract

Subspace learning techniques for text analysis, such as latent semantic indexing (LSI), have been widely studied in the past decade. However, to our best knowledge, no previous study has leveraged the rank information for subspace learning in ranking tasks. In this paper, we propose a novel algorithm, called learning latent semantics for ranking (LLSR), to seek the optimal latent semantic space tailored to the ranking tasks. We first present a dual explanation for the classical latent semantic indexing (LSI) algorithm, namely learning the so-called latent semantic space (LSS) to encode the data information. Then, to handle the increasing amount of training data for the practical ranking tasks, we propose a novel objective function to derive the optimal LSS for ranking. Experimental results on two SMART sub-collections and a TREC dataset show that LLSR effectively improves the ranking performance compared with the classical LSI algorithm and ranking without subspace learning.