Loosely-coupled mixed reality: Using the environment metaphorically

Abstract

Mixed reality refers to the (spectrum of) compositing of synthetic and real objects [2]. The two representative forms of mixed reality are the augmented reality (in which the content is mostly real), and augmented virtuality (mostly virtual). In both cases, the association between the real and virtual objects is usually desired to be as tight and explicit as possible. Thus, we can posit that the degree of this explicit registration will be one of the most important factors in the usability of mixed reality systems. Ironically, however, AR systems with high fidelity registration will also introduce significant usability problems due to the required special displays and sensors to be worn, in addition to the cost and computational difficulties. Therefore, in this paper, we newly propose a new form of mixed reality called the “Loosely coupled Mixed Reality (LMR),” which uses only minimal explicit and technologically supported association cues between the real and virtual objects. Instead, through intuitive metaphor between the virtual space and real operating environment, the burden of registration (fusing of the worlds) is put more on to the user. As such an approach is expected to introduce significant amount of cognitive load, an intuitive and metaphoric mapping between the two realities would be essential. Figure 1 illustrates the concept of LMR. The user is given two separate realities: one is the physical reality in which he is playing a golf game (in this case with the help of a hand-held device), and the other is the virtual golf environment presented through the hand-held device. The user starts by establishing a simple mapping in one's mind e.g. between objects in the physical reality (e.g. trashcan, street) and those in the game environment (e.g. green, golf course). This metaphor is intuitive because, for one, it is established by the user himself. Note that, to take full advantage of the environment metaphor, LMR becomes inherently mobile in its nature. In this example, as the user hits the ball and approaches the green/hole by physically moving in the corresponding real space, one's movement and interaction is reflected into the virtual world at the same time (through tracking). While the user is certainly aware of one's own actions, the user can “time to time” check upon the helping hand-held device to view the overall status (e.g. scores, other player position).