Heterogeneous modeling of gesture-based 3D applications

Abstract

Model-driven software engineering (MDE) is a well-known approach for developing software. It reduces complexity, facilitates maintenance and allows for the simulation, verification, validation and execution of software models. In this article, we show how MDE and model execution can be leveraged in the context of human-computer interaction (HCI). We claim that in this application domain it is beneficial to use heterogeneous models, combining different models of computation for different components of the system. We report on a case study that we have carried out to develop an executable model of a gesture-based application for manipulating 3D objects, using the Kinect sensor as input device, and the OGRE graphical engine as output device for real-time rendering. The interaction part of this application is fully specified as an executable heterogeneous model with the ModHel'X modeling environment. We exploit the semantic adaptation between different models of computation to implement a layered application using the most appropriate models of computation for each layer.