Simulated work tasks: the case of professional users

Abstract

This paper investigates simulated work tasks as a tool for information retrieval (IR) evaluation in a work-based, specialized setting. It has been shown that simulated work tasks must be tailored toward the group of study participant to ensure that the depicted situations are realistic and interesting from the participants' point of view [3]. Specifically, we investigate what characterizes an effective simulated work task in a professional government setting and how to design workable tasks for the evaluation of in-house information systems such as a corporate Intranet. The findings reveal that the test participants adopt the tasks. To understand and apply the simulated work tasks, the participants draw on different types of experience and knowledge: Topical, related, structural, and common knowledge. The study also shows that the knowledge types identified are more important for successful retrieval of information than similarity of the simulated work tasks with the participants' daily work tasks.