Short-term methodology for long-term usability

Abstract

Approaches to understanding usability of computer interfaces over the long term typically rely on longitudinal studies, which are limited in scope to the period of the experiment. In this study, we explore whether a non-longitudinal, cross-sectional approach can reliably detect useful differences in usability between novices and experts. Our approach takes a "snapshot" of usability problems and behaviors across a heterogeneous sample of users, ranging from novice to expert. Our analysis suggests that a cross-sectional methodology can distinguish between less experienced and more experienced users with respect to the kinds of applications that cause frustration, frequency of use of help, and whether the problem was solved. Our analysis also suggests that the method is poor at distinguishing causes of frustration and the overall distribution of types of solutions tried. The data also suggest that three months of use of an application is the most useful point at which to distinguish less-experienced from more-experienced users.