Aggregating Subjective and Objective Measures of Web Search Quality using Modified Shimura Technique

Web searching is perhaps the second most popular activity on Internet. Millions of users search the Web daily for their purpose. But as there are a number of search engines available, there must be some procedure to evaluate them. In this paper, we try to present an effort in this regard. For subjective measure, we are taking into account the "satisfaction " a user gets when presented with search results. The feedback of the user is inferred from watching the actions of the user on the search results presented before him in response to his query. For objective measures, we use Vector space model and Boolean similarity measures. All the three measures are aggregated using modified Shimura technique of rank aggregation. The aggregated ranking is then compared with the original ranking given by the search engine. The correlation coefficient thus obtained is averaged for a set of queries. We show our experimental results pertaining to seven public search engines and fifteen queries.