Concept-based Web Search using Domain Prediction and Parallel Query Expansion

Abstract

We address the problem of irrelevant results for short queries on Web search engines using latent semantic indexing in the WordSpace model and query expansion. First, we predict the potential concept topics, which are the domains for the search terms. Next, we expand the search terms in each of the predicted domains in parallel. We then submit separate queries, specialized for each domain, to a general-purpose search engine. The user is presented with categorized search results under the predicted domains. We prepared a categorized text collection (corpus) using Web directory listing to build word association models. We compare the results obtained using this corpus with those using Reuters corpus. User evaluations indicate that our approach helps the users avoid having to examine irrelevant Web search results, especially with short queries