Optimising Free Hand Selection in Large Displays by Adapting to User's Physical Movements

Advance in motion sensing technologies such as Microsoft Kinect and ASUS Xtion has enabled users to select targets on a large display through natural hand gestures. In such interaction, the users move left and right to navigate the display, and they frequently adjust body proximity against the display thus to switch between overall views and focus views. These physical movements benefit information navigation, interaction modality switch, and user interface adaptation. But in more specific context of free hand selection in large displays, the effect of physical movements is less systematically investigated. To explore the potential of physical movements in free hand selection, a physical movements-adapted technique is developed and evaluated. The results show that the new technique has significant improvements in both selection efficiency and accuracy, the more difficult selection task the more obvious improvement in accuracy. Additionally, the new technique is preferred to the baseline of pointer acceleration (PA) technique by participants.