Challenges for evaluating mobile systems with moving users

The major challenge for usability tests on mobile systems is to reproduce the context in which they are used. This paper points out some of the main difficulties encountered in the evaluation of such mobile systems, considering the real context of the user. Considering the need to assess such applications on the user's context, it was conducted a field study to evaluate an urban mobility system and, for this purpose, it was proposed a collection instrument, which can be used by the observer to take notes related to the evaluation, enabling various forms of content (observations, important points of the environment, user's verbal utterances, etc.). Through the field study, it was evaluated the benefits of using the proposed instrument, as well as identifying some of the difficulties faced in this type of study, i.e., evaluation of mobile systems on the move.