dSensingNI: a framework for advanced tangible interaction using a depth camera

Abstract

This work presents an approach to overcome the disadvantages of existing interaction frameworks and technologies for touch detection and object interaction. The robust and easy to use framework dSensingNI (Depth Sensing Natural Interaction) is described, which supports multitouch and tangible interaction with arbitrary objects. It uses images from a depth-sensing camera and provides tracking of users fingers of palm of hands and combines this with object interaction, such as grasping, grouping and stacking, which can be used for advanced interaction techniques.