Tangible Interaction: A case for light-weight tangibles

Abstract

The allure of using everyday objects to interact with virtual systems has been around since the beginnings of HCI. There have been many exciting research projects. Without exception they report increase user engagement and a fun user experience; positive attributes for learning and gaming systems. A few of the projects have even made it to products. However there are still major challenges to overcome in order to be able to use any everyday object as an interactive device. There appear to be some cognitive advantages to manipulating physical objects. However these advantages are not yet well understood and more research is needed to flesh out our understanding. In the meantime we can posit that the advantages come from the increased sensory engagement -- tactile, proprioceptive and kinesthetic, the true 3D visual space, our social norms of collaboration and turn-taking and our understanding of the physical world. As an example, recall when you have been sitting with a friend, colleague or child and one of you has grabbed the salt and pepper to represent the sun and earth, or such. Today and alternative might be to pull out a tablet and look for an animation. The animation will likely be more accurate, but less engaging. Which would be understood better, remembered more clearly? Could a blend of the two, with the salt and pepper used as tokens to manipulate the virtual animation, be built? Of course we could do this with current technologies, just not in the cafe. To sense the salt and pepper we could use image processing. This needs a minimum of two cameras to accurately sense the position and orientation. However picking up and moving the objects partially obscures them making tracking more difficult. A better set up is about 4 cameras -- and well behaved participants. Alternatively we could sense the objects on or near to a surface using a range of sensing technologies. The limitations with the surface sensing approach are distance from the surface and fidelity of the positioning. We could blend cameras and surface sensing. Is near field radio waves a possibility? The second challenge is how and where to provide feedback to the user on the effect of their actions on the virtual world. Should we show the world on a 2D display, through augmented or virtual reality headsets, have actuated tangibles that will travel the paths for the sun and earth? Clearly, to do this in the wild with current technologies, the setup time would be considerable and the moment would be lost. The challenge with light-tangibles is to be able to play out the scenario above without spending a 'month of Sundays' designing and building the system and going to the lab to test it. In this talk I will explore the challenges and possibilities of light-weight tangible interaction. Will there be a day when we can simply pick up the salt and pepper and play out the scenario?