Context effect on query formulation and subjective relevance in health searches

Abstract

It is recognized by the Information Retrieval community that context aects the retrieval process. Query formulation and relevance assessment are stages where the user role is central. The first determines what the system will search for and the second is frequently used to evaluate how the system behaved. With a large human involvement, these stages are expected to be largely influenced by user and task characteristics. To analyze the influence of these context features on the specified stages of health information retrieval, we conducted a user study in which we collected user features through two questionnaires. User characteristics include features like age, gender, web search experience, health search experience and familiarity with the medical topic. Task features include the medical specialty, the question type, the task's clarity and the task's easiness. Besides user and task features, the relevance assessment analysis also covered features related to the query and document. We found many variables do indeed aect query formulation and relevance judgment. Some of our results question evaluations using test collections and ask for evaluation models that incorporate other kind of success measures.