User interviews revisited: identifying user positions and system interpretations

Abstract

Throughout the history of HCI, interviews have been utilized for collecting users' subjective evaluations of interactive technology. This paper raises the issue that these interviews are often deployed in a manner overlooking two aspects of evaluation: the relative positions from which the system is evaluated and the interviewees' interpretations of the system. In the study, 14 users of a new information system were asked to evaluate provocative claims about the system's usability. The analyses of their responses reveal two sources of variation: what is being evaluated and who is evaluating it. Interviewees evaluated the system's usability from five user positions: end user, supervisor, organization's representative, co-developer, and outsider. Also, four "faces" of the system were interpreted: UI, utility, communication medium, and unknown entity. These findings are employed for drawing of broader conclusions about the system and its use, and procedures for improving user interviews in HCI are presented.