Evaluating a tool for the exploratory analysis of usability information using a cognitive walkthrough method

Results of empirical usability evaluations in large software-development organizations constitute a valuable asset for these companies. Information needs of usability professionals in these organizations are diverse, and involve both qualitative findings and quantitative data from diverse research methods and sources. Therefore, usability specialists need support for organizing, retrieving, assessing, and analyzing the internal results of usability research. This paper focuses on a method used to evaluate a prototype of a faceted retrieval tool that specifically supports usability specialists accessing a collection of usability results. The evaluation (n=11) was conducted using a primarily qualitative, scenario-based approach. Because of this, it was possible to direct evaluation towards conceptual issues instead of examining details of the surface of the interface. In addition, a survey collected answers to standardized items on the usefulness and ease of use of the system in combination with more domain specific questions. Together, these results provide a valid foundation for the assessment of the usefulness and the relative priority of features.