Evaluating e-books

In this paper, we discuss the importance of evaluation and in particular the impact of user studies on the progress and improvement of e-books and their usability, with particular attention to their design and functionalities aimed at increasing overall user experience. While there is a consensus on the importance of good design and few authors have already been publishing guidelines and provided advice on good practice, still no much attention has been paid to evaluation and its impact on e-book quality. It would indeed be extremely useful for designers to have a common platform in terms of bench-marks, agreed procedures, criteria and measures to evaluate the impact of their products on users but this is far from happening. The reality is that quite a few researchers are setting up evaluation experiments and collecting evidence of good practice but mostly in isolation from other previous or current similar initiatives. Under the INEX Book Track a specific task called Active Reading Task (ART) has been designed to gather efforts across the e-book community. Over the last two years we have tried to raise awareness in related communities but we are still facing serious problems in terms of participation. Indeed, we understand how demanding taking part in ART maybe, as working with users, real or surrogates, always has a high cost in terms of time and resources, this is why we are working to make the overall experience as rewarding, efficient and painless as possible. We strongly believe that such initiative would bring lots of benefits to the e-book community at large, in terms of improved design and quality of e-books. This paper starts with a description of ART, as part of the INEX initiative. We will then point out known problems with participation and discuss how to increase visibility, awareness and participation to ART by involving as many members of relevant communities as possible and by providing them with support and examples of good practice.