Application Research on Latent Semantic Analysis for Information Retrieval

Abstract

The basic principle of Classic traditional information retrieval model is the machine matching of the key word, namely retrieval based on keywords. This paper proposes a pre-clustering-based latent semantic analysis algorithm for document retrieval. The algorithm can solve the problem of time consuming computation of the similarity between the query vector and each text vector in the traditional latent semantic algorithm for document retrieval. It first clusters the documents using k-means clustering based on the latent semantic analysis, finds out the central point of each cluster, and then calculates the similarity between the query vector and each cluster's central points for retrieval. In view of the characteristics of document retrieval, it proposes a new method for calculating the feature weights and adopts the method of pre-clustering to preprocess document collection. The results of the experiment show that the new algorithm can reduce the search time, and improve the retrieval efficiency.