DRDLC: Discovering Relevant Documents Using Latent Dirichlet Allocation and Cosine Similarity

Abstract

In recent years, the availability of digital documents over web is increased drastically and there is a need for effective methods to retrieve and organize the digital documents. Since data is dispersed globally and is unorganized, it is a challenging task to develop an effective methods that can generate high quality features in these documents. It is necessary to reduce the gap between users search intention and the retrieved results known as semantic gap. In this paper, Discovering Relevant Documents using Latent Dirichlet Allocation and Cosine Similarity (DRDLC) is proposed. Word similarity is computed using CS Cosine Similarity present in search results documents. LDA is applied on extracted patterns and documents. Hashing is used to extract high relevant documents efficiently. Further, term synonyms are identified using word net and the documents are re-ranked. Experiments using the model Relevance Feature Discovery (RFD) on Reuters Corpus Volume-1 (RCV-1) show that the proposed DRDLC framework results in improved performance by providing more relevant documents to the user input query.