XDKinect: development framework for cross-device interaction using kinect

Abstract

Interactive systems set in multi-device environments continue to attract increasing attention, prompting researchers to experiment with emerging technologies. This paper presents XDKinect--a lightweight framework that facilitates development of cross-device applications using Kinect to mediate user interactions. The main benefits of XDKinect include its simplicity, adaptability and extensibility based on a flexible client-server architecture. Our framework features a time-based API to handle full-body interactions, a multi-modal API to capture gesture and speech commands, an API to utilise proxemic awareness information, a cross-device communication API, and a settings API to optimise for particular application requirements. A study with developers was conducted to investigate the potential of these features in terms of ease of use, effectiveness and possible use in the future. We show several example applications of XDKinect, as well as discussing advantages and limitations of our framework as revealed by our user study and experiments.