AR typing interface for mobile devices

We propose a new user interface system for mobile devices. By using augmented reality (AR) technology, the system overlays virtual objects on real images captured by a camera attached to the back of a mobile device, and the user can operate the mobile device by manipulating the virtual objects with his/her hand in the space behind the mobile device. This system allows the user to operate the device in a wide three-dimensional space and to select small objects easily. Also, the AR technology provides the user with a sense of reality in operating the device. We developed a typing application using our system and verified the effectiveness by user studies. The results showed that more than half of the subjects felt that the operation area of the proposed system is larger than that of a smartphone and that both AR and unfixed key-plane are effective for improving typing speed.