"Beyond the desktop, Out of the Office..." Designing Interactive Graphics Applications for Mobile Devices

The recent availability of increasingly powerful PDAs and mobile phones is making it possible to develop new applications based on interactive (2D as well as 3D) graphics. However, limitations of mobile devices and the peculiar needs of users on-the-go require a careful design of applications that are specifically thought for mobile devices and users [Chi06]. Mobile graphics applications become even more innovative and provide functionalities that were unavailable on desktop systems when they are integrated with various sensors (e.g., GPS, accelerometers, heart rate monitors, pulseoximeters, ...) that allow one to adapt the behavior of the application according to position in space (location-awareness) and other parameters (context-awareness). In this way, the mobile device becomes able to choose what to draw and how to draw it on the display based on what is happening to the user as well as the physical world that surrounds her. Moreover, the data recorded by mobile sensors about the behavior of users on the move lends itself to the design of visualizations that can be important in domains such as health care, urban planning, geomarketing, emergency management, and others. This invited talk will first discuss why and how mobile graphics applications require to be approached differently from traditional ones. Then, it will demonstrate some new mobile applications that have been developed in domains as diverse as tourism, health and fitness, navigation, and architectural visualization. Finally, it will deal with the visualization of mobile users’ behaviors by showing two different tools we have recently proposed [CRI06, NC07]. 1. Outline of the Talk Mobile devices (such as PDAs and mobile phones) are becoming more and more widespread, presenting designers of graphics applications with new challenges and opportunities. First, the devices have become powerful enough to support new, mobile applications of interactive (2D as well as 3D) graphics, but limitations of mobile devices and the peculiar needs of users on-the-go require a careful design that specifically addresses mobile devices and users [Chi06]. Second, these applications can be augmented with sensors that allow designers to make them context-aware as well as record information (e.g., position, orientation, physiological parameters,...) about their users. Therefore, visualizing information about the recorded mobile users’ behavior is another new subject that will become increasingly relevant in domains such as health care, urban planning, geomarketing, emergency management, user studies, and others. The keynote speech will discuss the different issues involved in designing interactive graphics applications for mobile devices, as well as present examples of applications for mobile devices and tools for the visualization of information about mobile users’ behaviors [CRI06, NC07]. The talk is organized in three parts that are described in more detail in the following. c © The Eurographics Association 2007. Eurographics Italian Chapter Conference (2007) Raffaele De Amicis and Giuseppe Conti (Editors) Luca Chittaro / Designing Interactive Graphics Applications for Mobile Devices 1.1. Doing Graphics on Mobile Devices Computer graphics has a relevant role in almost every domain of computer applications. It is thus natural to think about bringing it to mobile devices to exploit the power of graphics anytime, anywhere. Unfortunately, limitations of mobile devices make it impossible to follow a trivial porting approach from desktop computers. A considerable research effort is needed to understand how to design and implement effective graphics applications for mobile devices. This invited talk will first discuss in detail the peculiarities of the mobile graphics context that motivate research. Mobile devices have limitations that are not typical of desktop computers and the mobility context is different in many ways from that of traditional computer graphics. The speech will illustrate the differences that have to be taken into account in developing a mobile graphics application. It will also mention issues concerning limited cognitive resources and safety [CD04] of mobile users as an additional motivation to employ mobile graphics effectively as a way to provide information at-a-glance that is easily understood with less cognitive resources and distracts the user as less as possible from her current primary task. Then, we will deal with the different issues of designing a mobile graphics application, especially focusing on the so-called presentation problem [AS01, BR03, BCCR04, BCG06, Chi06, RCSH04] since the limited screen size of mobile devices makes it particularly difficult. Figure 1: The MobiX3D player [NCB06] displays 3D content (described in the ISO standard X3D) on mobile devices; it can be downloaded from http://hcilab.uniud.it/MobiX3D Figure 2: The MAGDA system [BC05b] supports interactive visual queries of geo-referenced databases. 1.2. Interactive Graphics Applications For Mobile Users The talk will provide concrete examples of different mobile applications and the different classes of visualizations that are being investigated in the mobile context, i.e. Text (e.g., list of names, menus, e-books, documents,...); Pictures (e.g., photographs, figures, artwork,...); Maps (e.g, tourist maps, first-responder maps, architect maps,...); Physical objects (e.g., CAD models, interactive engineering instructions, scientific visualizations,...); Abstract Data (e.g, timeoriented data such as calendars, medical records, stock market data,...). Particular attention will be given to mobile 3D graphics, made possible by recent technologies such as Open GL ES [NCB06] (see Figure 1), and to interaction techniques such as visual queries [BC05b] (see Figure 2). Mobile graphics applications become even more innovative and provide functionalities that were unavailable on desktop systems when they are integrated with various sensors (e.g., GPS, accelerometers, heart rate monitors, pulseoximeters, ...) that allow designers to adapt the behavior of the application according to position in space (location-awareness) and other parameters (contextawareness). In this way, the mobile device becomes able to choose what to draw and how to draw it on the display based on what is happening to the user and to the physical world that surrounds the user. Some of the 3D graphics applications that exploit sensors give new meaning to the word "virtual" by mixing, relating or synchronizing the virtual world on the mobile device and the physical world that surrounds the user. Examples of such mobile 3D applications that will be discussed in this talk are: (i) mobile tourist guides or navigation systems that lead users to discover the real world by synchronizing it with the virtual world on the device [BC05a], (ii) health and fitness guides that monitor physiologic parameters and train the user by choosing and explaining the most appropriate exercises for her in the field [BCN06] (see Figure 3), (iii) virtual characters on mobile devices with various roles in supporting the user [CBN06]. 1.3. Visualizing Information About Mobile Users Employing the above mentioned sensors on mobile devices opens up new opportunities in the collection of temporal and spatial data about mobile users’ behaviors. From a computer graphics point of view, the data recorded by sensors about the behavior of users on the move lends itself to the design of visualizations that can be important in domains such as health care, urban planning, geomarketing, emergency management, and others. For example, visualizations that allow an analyst to understand how people navigate roads or buildings could play an important role in the design, evaluation and optimization of such spaces (e.g., planning transports or evacuation procedures), while visualizations of physiological parameters of a user on the move can help her in c © The Eurographics Association 2007. 152 Luca Chittaro / Designing Interactive Graphics Applications for Mobile Devices Figure 3: The MOPET system [BCN06] exploits contextaware 3D demonstrations of fitness exercises to train users in outdoor fitness trails. better understanding her fitness status (e.g., to better plan her training) or a physician in assessing her condition and giving more personalized advice or a professional trainer in comparing the performance of different athletes in a sport team. The talk will illustrate these new visualizations by using two different tools we have recently proposed [CRI06, NC07]. The VU-Flow [CRI06] tool provides a set of interactive visualizations that highlight interesting navigation behaviors of single or groups of mobile entities. Some of the proposed visualizations concern: i) detailed replay of users’ behavior, ii) areas where users spent more/less time, iii) more/less seen objects, iv) areas of flow congestion, and v) predominant users’ directions of movement. The MOPET Analyzer [NC07] tool provides instead a set of interactive visualizations for the visual analysis of geo-referenced fitness performance data, such as running speed and heart rate.